-- Logs begin at Thu 2022-07-21 21:43:40 UTC, end at Thu 2022-07-21 22:53:51 UTC. -- Jul 21 22:17:35.402351 dv-cinder-850698-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 22:17:35.411645 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 22:17:35 2022] *** Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: nodename: dv-cinder-850698-1-iscsi Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: machine: x86_64 Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: clock source: unix Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: pcre jit disabled Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: detected number of CPU cores: 4 Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: current working directory: / Jul 21 22:17:35.414247 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: detected binary path: /usr/bin/uwsgi-core Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: your processes number limit is 47680 Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: your memory page size is 4096 bytes Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: detected max file descriptor number: 2048 Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: lock engine: pthread robust mutexes Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: thunder lock: enabled Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 22:17:35.414940 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 22:17:35.458653 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: Python main interpreter initialized at 0x5628d033a9c0 Jul 21 22:17:35.458653 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: python threads support enabled Jul 21 22:17:35.458653 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: your server socket listen backlog is limited to 100 connections Jul 21 22:17:35.458653 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: your mercy for graceful operations on workers is 90 seconds Jul 21 22:17:35.459022 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 22:17:35.459117 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: *** Operational MODE: preforking *** Jul 21 22:17:35.459399 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: *** uWSGI is running in multiple interpreter mode *** Jul 21 22:17:35.459399 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: spawned uWSGI master process (pid: 80264) Jul 21 22:17:35.459584 dv-cinder-850698-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 22:17:35.460464 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: spawned uWSGI worker 1 (pid: 80276, cores: 1) Jul 21 22:17:35.460561 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: spawned uWSGI worker 2 (pid: 80277, cores: 1) Jul 21 22:17:35.460697 dv-cinder-850698-1-iscsi devstack@keystone.service[80264]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 22:17:41.035104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:17:41.035104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: from cryptography.utils import int_from_bytes Jul 21 22:17:41.036267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:17:41.036267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: from cryptography.utils import int_from_bytes Jul 21 22:17:41.036756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:17:41.036756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: from cryptography.utils import int_from_bytes Jul 21 22:17:41.037531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:17:41.037531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: from cryptography.utils import int_from_bytes Jul 21 22:17:41.086487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 22:17:41.086708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 22:17:41.086903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] command line args: [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 22:17:41.087230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 22:17:41.087427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ================================================================================ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 22:17:41.087708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] admin_token = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.087988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config_dir = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.088206 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.088424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config_source = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.088559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 22:17:41.088648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] debug = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.088791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 22:17:41.088925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.089247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] command line args: [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 22:17:41.089416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.089533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 22:17:41.089645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.089742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ================================================================================ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 22:17:41.089837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.089936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] admin_token = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_dir = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_config_append = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_source = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.090900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] debug = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.096691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_dir = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.096869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.097903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_config_append = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_dir = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.098980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.099930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.100008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.100122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.100388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.101022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.101022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.101974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] publish_errors = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.102961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] standard_threads = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103539 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_journal = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.103762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_json = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_stderr = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] publish_errors = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_syslog = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.104818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.104963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.105082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.105205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.105342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.105458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.105593 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.105712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] standard_threads = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.105832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.105948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.106067 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.106183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.106280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.106382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.106382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_journal = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.106575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.106669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_json = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.106781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.106897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_stderr = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.106897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_syslog = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.107133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:17:41.107474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.107966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.108951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.109986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.110901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.111987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.112919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.113863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.114984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.115803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.116008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.116156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.116951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.117978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.118899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.119984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.120918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.121872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.122956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.123983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.124919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.125968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.126936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.127955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.external = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.128985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.password = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.129994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.token = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.130991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.131949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.132299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.external = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.133932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.password = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.token = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.134932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.135937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.136996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.137950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.138888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.139711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.140963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.141972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.142920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.143875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.144918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.145934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.146876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.147005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.147756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.147843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.147917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.147990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.148255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.148792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.148890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.148969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.149954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150489 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.150862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.151983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.152976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.153985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.154964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.155964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.156928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.157979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.158905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.159975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.160953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.161979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.162921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.163769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.164159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.164241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.164598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.164725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.164848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.164979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.165939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.166001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.166069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.166166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.166245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.167854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.167991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.168992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.169940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.170968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.171994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.172998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 22:17:41.173296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.173839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.174092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.174411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.174729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.174983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.175251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.175494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.175734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.175975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.176221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.176460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.176714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.176958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.177212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.177457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.177701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.177941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.178185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.178457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:17:41.178753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 22:17:41.204196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.204549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.204771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.205638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.205979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.206190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.214251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.215846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.244367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.244772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.245664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.245878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.393197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.393531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.444541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.444879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.446572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.446814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.512309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.512532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.512713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.512802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.512952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.513804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.540422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.540737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.540824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.540921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.541817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.542031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.567151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.567688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568845 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.568974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.569282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.569511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.569740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.570063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:17:41.587471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.587674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:17:41.588044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.588208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:17:41.588468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.588729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.588820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.588964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:17:41.589082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.589219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.589343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.589550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:17:41.589871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.590210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.590600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.590862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.601705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.602014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.602289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.602575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.603819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.604196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.604391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.604696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.611422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.614479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.614643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.614761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.615048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.615476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.615897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.616277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.616277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.616936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.616936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.617174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.617297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.619282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.619596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.619842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.620459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.620855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.621234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.621581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.621866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.622144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.622423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.622698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.644849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.646382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.646656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.651982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.652296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.652579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.652854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.653638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.659300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.659582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.659864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.660195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.660483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.665986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.666264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.666542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.666819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.667108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.667377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.667638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.667778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.667898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.668064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.672915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.673201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.673483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.673768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.674050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.674343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.674634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.674940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.687314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.687610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.692326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.695313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.695493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.696515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.696815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.700071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.701898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.703769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.704064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.705505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.709107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.709271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.709567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.710579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.710945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.711354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.711558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.713337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.717169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.717469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.717762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.718049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.718346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.718625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.730514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.730899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.731191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.731454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.738067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.738466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.738738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.739010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.739360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.741134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.741402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.744467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.744743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.745023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.745398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.745674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.747334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.749051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.749323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.752429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.752712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.752994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.753272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.753545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.759608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.759889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.760164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.760437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.760709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.760990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.761260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.767883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.768168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.768458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.768735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.769019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.769310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.769583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.781572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.784876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.788250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.789937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.790333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.791616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.791885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.792163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.793676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.797029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.798625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.799619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.800275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.800555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.800844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.801228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.805061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.805340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.805617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.805893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.809026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.810658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.813890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.814179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.814464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.814746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.815100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.815399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.815668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.824284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.824564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.824665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.824839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.824948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.825108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.825192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.825374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.825647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.825924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.826195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.826465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.826727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.826990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.827268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.827536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.827809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.833136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:17:41.833416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.833698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.833972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.834241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.834511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.834937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.835275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.835488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.835757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.836038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.836301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.836562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.836828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:17:41.865751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5628d033a9c0 pid: 80276 (default app) Jul 21 22:17:41.869863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de868ade-df74-4a12-bd1a-4148f3666023 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:41.870156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de868ade-df74-4a12-bd1a-4148f3666023 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:41.870434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de868ade-df74-4a12-bd1a-4148f3666023 None None] PATH_INFO: `/v3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:41.875198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Thu Jul 21 22:17:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:17:41.875440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5628d033a9c0 pid: 80277 (default app) Jul 21 22:17:49.759707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:49.760379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:49.761024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:49.762988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 2/2] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:49.773370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:49.773913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:49.774010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:49.852070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: INFO dbcounter [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] Registered counter for database keystone Jul 21 22:17:49.875724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG oslo_db.sqlalchemy.engines [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 22:17:49.887234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writer thread running {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 22:17:50.089741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG passlib.handlers.bcrypt [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80277) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 22:17:50.090165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG passlib.handlers.bcrypt [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80277) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 22:17:50.148891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:50.150914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:50.300659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1/3] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 22:17:50.464688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:50.542675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: INFO dbcounter [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] Registered counter for database keystone Jul 21 22:17:50.563961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG oslo_db.sqlalchemy.engines [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80276) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 22:17:50.571072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writer thread running {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 22:17:50.702353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aee50-2245-44b8-b4c1-5e343fad9da7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:50.713407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:50.714169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:50.714463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:50.714830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:50.724166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:50.781014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 22:17:50.781687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496ed1-c3d6-4bd6-aab9-9f8b8bb960bf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:50.793637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 3/4] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Jul 21 22:17:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 22:17:50.799840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:50.815911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df8cb8-4f68-4e76-a3a0-11c297ec4b63 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:50.820666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:50.821825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:50.822118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:50.822401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] PATH_INFO: `/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:50.826746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] RBAC: Authorizing `identity:get_project(project_id=1a2e5fa7e325464b9d8cfafd39af8ec6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:50.878224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 22:17:50.878572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd623039-ce3c-4cd4-9905-023d7572b463 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:50.883582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 2/5] 10.222.0.58 () {56 vars in 1207 bytes} [Thu Jul 21 22:17:50 2022] GET /identity/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:17:52.982391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a246772-465d-4d79-bd2a-598252d8c6d7 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:52.982906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a246772-465d-4d79-bd2a-598252d8c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:52.982906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a246772-465d-4d79-bd2a-598252d8c6d7 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:52.986971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 4/6] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:52.993405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:52.993873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:52.993979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:53.012554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:53.014107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:53.022174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 3/7] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 22:17:53.045337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a246772-465d-4d79-bd2a-598252d8c6d7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:53.058274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a246772-465d-4d79-bd2a-598252d8c6d7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:53.062761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c7efb-c9bc-4673-b2b9-f2b5764575e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:53.063438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7efb-c9bc-4673-b2b9-f2b5764575e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:53.063718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7efb-c9bc-4673-b2b9-f2b5764575e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:53.063992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c7efb-c9bc-4673-b2b9-f2b5764575e3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:53.072249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c7efb-c9bc-4673-b2b9-f2b5764575e3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:53.074038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c7efb-c9bc-4673-b2b9-f2b5764575e3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:53.084796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 5/8] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Jul 21 22:17:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:17:53.090971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:53.100939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c380d4-8fc2-40a1-9d1d-752d93dc2d6f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:53.105305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9764ce-68e9-4eaa-9419-d87995a78704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:53.106165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9764ce-68e9-4eaa-9419-d87995a78704 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:53.106448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9764ce-68e9-4eaa-9419-d87995a78704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:53.106724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9764ce-68e9-4eaa-9419-d87995a78704 admin admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:53.108145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9764ce-68e9-4eaa-9419-d87995a78704 admin admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:53.109994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9764ce-68e9-4eaa-9419-d87995a78704 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:53.111604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 4/9] 10.222.0.58 () {56 vars in 1198 bytes} [Thu Jul 21 22:17:53 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:17:55.245915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c247d1-627c-4290-8c4f-60346c5038da None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.246440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c247d1-627c-4290-8c4f-60346c5038da None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.246547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c247d1-627c-4290-8c4f-60346c5038da None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.248008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 6/10] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:55.249127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2cb3e-5976-4be8-9283-b6dc1b1c7534 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.249501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2cb3e-5976-4be8-9283-b6dc1b1c7534 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.249691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2cb3e-5976-4be8-9283-b6dc1b1c7534 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.251043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 5/11] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:17:55.252878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.253163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.253437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.256187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.256484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.256866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.273965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:55.275205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:55.294469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG passlib.handlers.bcrypt [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80276) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 22:17:55.294892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG passlib.handlers.bcrypt [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80276) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 22:17:55.342993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:55.344649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:55.366924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:17:55.398220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:55.425024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:55.456211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:17:55.479154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:55.514377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb82aa2-b8c8-4a20-87e5-7092215085e3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:55.549576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:55.550378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.550664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.550942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.562752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:17:55.574897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:17:55.575861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:17:55.576633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:17:55.580257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99568fb9-dccd-41d7-9882-23ed671a9691 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:55.581904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:55.582830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:55.620529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:55.621431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.621719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.621996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.668512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:55.689164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44af492-ae2b-426d-be9d-e0408ba73176 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:55.689724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:55.691305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:55.728615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:55.729288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.729622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.729851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.730606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:17:55.730913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:55.732381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:55.744782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:17:55.750324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:55.794607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 8/16] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:17:55.800683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:55.803531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:55.857172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96387aad-1aeb-4977-a94f-02ddb882cd66 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:55.898001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:55.898518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.898806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.899099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.900371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:55.902125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:55.903959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:17:55.909241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:55.910320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63094-94f8-4cba-bf3e-0c68297acc35 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:55.924440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:55.946047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76801b7-d700-43da-a761-9766df32654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:55.946859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76801b7-d700-43da-a761-9766df32654b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:55.947163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76801b7-d700-43da-a761-9766df32654b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:55.947442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76801b7-d700-43da-a761-9766df32654b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:55.967086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76801b7-d700-43da-a761-9766df32654b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:55.968664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76801b7-d700-43da-a761-9766df32654b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f76801b7-d700-43da-a761-9766df32654b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:17:55.991814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:17:55.992722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:17:55.993528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:17:55.993528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:17:55.993528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:17:55.993528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:17:55.993528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:17:55.998817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76801b7-d700-43da-a761-9766df32654b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:56.009981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379b92a-7304-4eb7-8164-6157b1218929 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:56.044202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:56.045063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:56.045290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:56.045690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:56.046764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:56.048256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:56.049862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:17:56.056347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:56.056816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:56.105628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76801b7-d700-43da-a761-9766df32654b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:56.142912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:56.146943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:56.147879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:56.148194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:56.148474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:56.149330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:17:56.149682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:56.151547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323a4b0-8324-40ab-9f1a-d7b9965f0026 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:56.169849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf6195-4a43-40c3-88c9-90d1bead3038 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:56.174264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:17:56.209236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:56.209928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:56.210210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:56.210484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:56.211379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.domain.id=default, user.id=c37a1ed5575d4f4ab7615ea38d6c47b0) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:17:56.211693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:56.213484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dfa11-08b7-416b-8744-8e4422b75151 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:56.221250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 11/21] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Jul 21 22:17:56 2022] GET /identity/v3/role_assignments?user.id=c37a1ed5575d4f4ab7615ea38d6c47b0&scope.domain.id=default&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:17:58.695239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aa823-8a10-4d17-8ee2-ac902751788e None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:58.696197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aa823-8a10-4d17-8ee2-ac902751788e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:58.696672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aa823-8a10-4d17-8ee2-ac902751788e None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:58.698561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:58.704056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e93733-0598-4581-87b7-0c41814387aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:58.706396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e93733-0598-4581-87b7-0c41814387aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:58.706909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e93733-0598-4581-87b7-0c41814387aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:58.736405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e2e93733-0598-4581-87b7-0c41814387aa None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:58.737957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e2e93733-0598-4581-87b7-0c41814387aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:58.834355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:17:58.869518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aa823-8a10-4d17-8ee2-ac902751788e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:58.879662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d519-0e3c-4999-8060-a0414fe26d38 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:58.883246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d519-0e3c-4999-8060-a0414fe26d38 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:58.883766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d519-0e3c-4999-8060-a0414fe26d38 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:58.886956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:58 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:58.893741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:58.894482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:58.897268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:58.934299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:58.936703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:59.129618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aa823-8a10-4d17-8ee2-ac902751788e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:59.136990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 14/25] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:17:59.180105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:59.184461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:59.185518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:59.226370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:59.227759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.228258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.228749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.267165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:59.270339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-afa15bad-bbe2-4cdb-b4e7-430a301c8e6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:17:59.298789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:17:59.300321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 12/26] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18624-fd43-4cb9-8275-2b64b21ef308 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18624-fd43-4cb9-8275-2b64b21ef308 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.302799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18624-fd43-4cb9-8275-2b64b21ef308 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.303884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 13/27] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:59.305444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed981d7-c375-449c-8496-56364236cf7d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.305444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed981d7-c375-449c-8496-56364236cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.305444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed981d7-c375-449c-8496-56364236cf7d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.308682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 14/28] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:17:59.309709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed981d7-c375-449c-8496-56364236cf7d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:59.325561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c079e-8c0b-449d-b8a8-cea8fc57cc80 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:59.369084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8484296-b6f7-4e6c-b156-e310c9b12536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:59.369778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8484296-b6f7-4e6c-b156-e310c9b12536 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.370227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8484296-b6f7-4e6c-b156-e310c9b12536 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.370512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8484296-b6f7-4e6c-b156-e310c9b12536 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.371647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8484296-b6f7-4e6c-b156-e310c9b12536 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:59.373098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8484296-b6f7-4e6c-b156-e310c9b12536 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:59.435202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed981d7-c375-449c-8496-56364236cf7d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:59.475574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:59.476249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.476540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.476817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.477568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:17:59.477880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:59.479366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674ee1d-a0f0-4f45-8c5c-620c8ca79875 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:59.492571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 15/29] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:17:59.494032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.494310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.494597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.507454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 15/30] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jul 21 22:17:59 2022] POST /identity/v3/roles => generated 219 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:17:59.508966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.509259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.509519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.513802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:59.515622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:59.529055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:17:59.530588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:17:59.553063 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:59.599767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:17:59.622497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 16/31] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:17:59.623401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:59.624895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 16/32] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:17:59.678784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:59.759307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec098a-ba08-4e60-8575-ed3d65a8904f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:59.797698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:59.798374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.798659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.798933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.800006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:17:59.801493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:17:59.863729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c231cd-f70c-40b6-ba87-f41419abb4aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:17:59.903850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:17:59.904713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:17:59.905004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:17:59.905284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:17:59.917373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:17:59.925298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:17:59.927462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:17:59.930434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 17/33] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:17:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:17:59.940326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:59.961083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 17/34] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:17:59 2022] POST /identity/v3/projects => generated 301 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:17:59.962209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:17:59.983149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:00.078489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:00.094297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d7b46-7225-4116-bdf6-45eb7d1dfb42 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:00.137371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:00.138253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.138669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.139073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.139989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:00.140500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:00.147089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:00.158314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 18/35] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:17:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:00.164060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:00.190602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b829-9f8c-446a-a6fa-9aaf1ccb0042 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:00.234209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:00.235355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.235743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.236072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.236942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:00.238509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:00.245697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:00.260482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 18/36] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jul 21 22:17:59 2022] POST /identity/v3/roles => generated 225 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:00.279806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36239d28-fcd1-4d70-8d07-bd7c49d172f1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:00.323026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62bbc6-7cbb-4e27-9503-f6c58205ed44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:00.324001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62bbc6-7cbb-4e27-9503-f6c58205ed44 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.324294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62bbc6-7cbb-4e27-9503-f6c58205ed44 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.324578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62bbc6-7cbb-4e27-9503-f6c58205ed44 None admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.326429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62bbc6-7cbb-4e27-9503-f6c58205ed44 None admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:00.327984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62bbc6-7cbb-4e27-9503-f6c58205ed44 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:00.330249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 19/37] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:00 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:00.336081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:00.470171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01651-59f4-46ec-8b4a-21486ff518be None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:00.488470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad949c-d8cc-4375-8906-a86b2572e4cc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.488765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad949c-d8cc-4375-8906-a86b2572e4cc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.489055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad949c-d8cc-4375-8906-a86b2572e4cc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.490129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:00.491339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 20/38] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:00.497510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.497806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.498100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.515242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:00.516061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.516348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.516627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.517834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:00.519359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:00.521257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 19/39] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:00.526655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:00.527306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:00.529833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:00.606700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 21/40] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:00.631459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:00.741522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:00.763030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa198793-5dc2-4994-abba-31b52b7c9fd7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:00.780785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b1f26-efa3-4e96-a0be-848cf21b35bc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:00.806920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:00.807844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.808133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.808419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] PATH_INFO: `/v3/domains/default/users/c37a1ed5575d4f4ab7615ea38d6c47b0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.810602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c37a1ed5575d4f4ab7615ea38d6c47b0, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:00.812108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:00.827679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658ec06-cd47-48b6-b2eb-f31ab0943a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:00.828495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ec06-cd47-48b6-b2eb-f31ab0943a03 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:00.828786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ec06-cd47-48b6-b2eb-f31ab0943a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:00.829104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ec06-cd47-48b6-b2eb-f31ab0943a03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:00.839224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 20/41] 10.222.0.58 () {58 vars in 1361 bytes} [Thu Jul 21 22:18:00 2022] PUT /identity/v3/domains/default/users/c37a1ed5575d4f4ab7615ea38d6c47b0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:00.840301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658ec06-cd47-48b6-b2eb-f31ab0943a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:00.841917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658ec06-cd47-48b6-b2eb-f31ab0943a03 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:00.849975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 22/42] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:00.851179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:00.903876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:00.982619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e04ca6-68f0-4240-9ce2-b4d579c5b2d5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:01.037128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92123892-f9dd-43c1-b22a-3307285016b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:01.038000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92123892-f9dd-43c1-b22a-3307285016b4 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:01.038415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92123892-f9dd-43c1-b22a-3307285016b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:01.038820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92123892-f9dd-43c1-b22a-3307285016b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:01.039759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92123892-f9dd-43c1-b22a-3307285016b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:01.041439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92123892-f9dd-43c1-b22a-3307285016b4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:01.075091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 21/43] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:18:00 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:01.278461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a00ee-c317-4e76-9ca9-a055a68fb648 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:01.278804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a00ee-c317-4e76-9ca9-a055a68fb648 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:01.278997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a00ee-c317-4e76-9ca9-a055a68fb648 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:01.280546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 23/44] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:01.285624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b7b1f-1fd2-450c-808e-fa60345100fd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:01.285927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b7b1f-1fd2-450c-808e-fa60345100fd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:01.286217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b7b1f-1fd2-450c-808e-fa60345100fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:01.305402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a76b7b1f-1fd2-450c-808e-fa60345100fd None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:01.306663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a76b7b1f-1fd2-450c-808e-fa60345100fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:01.383712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 22/45] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:01.407743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a00ee-c317-4e76-9ca9-a055a68fb648 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:01.511638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a00ee-c317-4e76-9ca9-a055a68fb648 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:01.546898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41082921-9f2c-4a64-bd9c-3fe9c6aea62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:01.547526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41082921-9f2c-4a64-bd9c-3fe9c6aea62a None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:01.547815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41082921-9f2c-4a64-bd9c-3fe9c6aea62a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:01.548097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41082921-9f2c-4a64-bd9c-3fe9c6aea62a None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:01.548864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41082921-9f2c-4a64-bd9c-3fe9c6aea62a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:01.550269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41082921-9f2c-4a64-bd9c-3fe9c6aea62a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:01.560613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:01.568732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 24/46] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jul 21 22:18:01 2022] POST /identity/v3/roles => generated 223 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:02.923389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b47b36-69df-4593-8062-00e7b3965dd3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:02.923894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b47b36-69df-4593-8062-00e7b3965dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:02.923894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b47b36-69df-4593-8062-00e7b3965dd3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:02.925264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 23/47] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:02.930880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc0df-6822-43c1-8be3-e1bfa95e8678 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:02.931331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc0df-6822-43c1-8be3-e1bfa95e8678 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:02.931617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc0df-6822-43c1-8be3-e1bfa95e8678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:02.949254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ff9fc0df-6822-43c1-8be3-e1bfa95e8678 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:02.950462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff9fc0df-6822-43c1-8be3-e1bfa95e8678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:03.018114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 25/48] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:03.043367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b47b36-69df-4593-8062-00e7b3965dd3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.068738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:03.122626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0441-f576-46c6-87ac-42524f20f38c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.122914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0441-f576-46c6-87ac-42524f20f38c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.123234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0441-f576-46c6-87ac-42524f20f38c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.125077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 26/49] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:03.130060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.130354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.130649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.147789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:03.149010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:03.150036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b47b36-69df-4593-8062-00e7b3965dd3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.185469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.186187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.186476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.186754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.198477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:03.206833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:03.207814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:03.209197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 24/50] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:03.214939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.223590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 27/51] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:03.247070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.307142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:03.311707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:03.325634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898f30-2178-4e1d-b7d7-90e290f16cd1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.366838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3545c5f-b023-447e-b66d-2e08228b1cbf None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.370868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.371597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.371887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.372165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.372933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:03.373246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:03.374721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:03.386221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 25/52] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:03.392371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.513252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.513980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.514268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.514550 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.515824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:03.517335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:03.519042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 28/53] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:03.524757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee98e-7fba-4d51-97ee-dd719e170a02 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.526021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.560962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.561862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.562151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.562435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.563702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:03.565184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:03.566850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 26/54] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:03 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:03.573225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.590162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:03.632767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359d8b8-d7d2-4764-b05b-96c5ad066714 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.661556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:03.671299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.671981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.672266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.672549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.673264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:03.674717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:03.683042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfeb50-6361-41f8-b66c-32ce2215fb9f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.704209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 29/55] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Jul 21 22:18:03 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:18:03.722036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2c352-ae58-4462-b1c6-f8583d56f0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.722964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2c352-ae58-4462-b1c6-f8583d56f0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.723363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2c352-ae58-4462-b1c6-f8583d56f0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.723711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2c352-ae58-4462-b1c6-f8583d56f0a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.725918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2c352-ae58-4462-b1c6-f8583d56f0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:03.727682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2c352-ae58-4462-b1c6-f8583d56f0a3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:03.729489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 27/56] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:03.736168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:03.856709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29638eaf-d4df-4ab3-8c38-0bce6157787b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:03.891882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:03.892558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:03.892858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:03.893139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:03.893997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.domain.id=default, user.id=c37a1ed5575d4f4ab7615ea38d6c47b0) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:03.895100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:03.896518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250126c-4ab5-4cff-93b4-ffbc5a184b32 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:03.906387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 30/57] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Jul 21 22:18:03 2022] GET /identity/v3/role_assignments?user.id=c37a1ed5575d4f4ab7615ea38d6c47b0&scope.domain.id=default&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 530 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:05.783636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2dbf-9a8a-4c22-9559-a461c6a49b34 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:05.784207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2dbf-9a8a-4c22-9559-a461c6a49b34 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:05.784207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2dbf-9a8a-4c22-9559-a461c6a49b34 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:05.785611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 28/58] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:05.791653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:05.792191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:05.792274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:05.810921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:05.812200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:05.848898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:05.881097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 31/59] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:05.906150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2dbf-9a8a-4c22-9559-a461c6a49b34 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:05.988463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:06.007301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2dbf-9a8a-4c22-9559-a461c6a49b34 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:06.041467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dad7e4-ee42-4341-93ff-dc91cb331c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:06.042250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dad7e4-ee42-4341-93ff-dc91cb331c3e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:06.042534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dad7e4-ee42-4341-93ff-dc91cb331c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:06.042813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dad7e4-ee42-4341-93ff-dc91cb331c3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:06.043998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dad7e4-ee42-4341-93ff-dc91cb331c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:06.045476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dad7e4-ee42-4341-93ff-dc91cb331c3e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:06.047122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 29/60] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:06.052711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:06.148760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c638672-f0c3-45ea-8201-7a5d9bed508c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:06.182649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4bc52-a974-427f-b2c4-b56212499195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:06.183351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4bc52-a974-427f-b2c4-b56212499195 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:06.183637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4bc52-a974-427f-b2c4-b56212499195 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:06.183914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4bc52-a974-427f-b2c4-b56212499195 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:06.184630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4bc52-a974-427f-b2c4-b56212499195 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:06.186102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4bc52-a974-427f-b2c4-b56212499195 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:06.212537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 32/61] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:06 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:09.203906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bfe53-cde1-48d3-ab67-56df3b44375d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.204930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bfe53-cde1-48d3-ab67-56df3b44375d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.205426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bfe53-cde1-48d3-ab67-56df3b44375d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.207118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 30/62] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:09.213031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.213930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.214413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.240910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:09.242472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:09.272494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:09.334226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 33/63] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:09.374839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bfe53-cde1-48d3-ab67-56df3b44375d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:09.514500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bfe53-cde1-48d3-ab67-56df3b44375d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:09.568826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ffa4-54b9-4182-8e77-17455873766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:09.569643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ffa4-54b9-4182-8e77-17455873766e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.569930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ffa4-54b9-4182-8e77-17455873766e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.570212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ffa4-54b9-4182-8e77-17455873766e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.588976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04ffa4-54b9-4182-8e77-17455873766e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8c04ffa4-54b9-4182-8e77-17455873766e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:09.596810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:09.597659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:09.598415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 31/64] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:09.604691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:09.692388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8251fb-78c3-4525-bef5-5cf286ee313d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.692692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8251fb-78c3-4525-bef5-5cf286ee313d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.692959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8251fb-78c3-4525-bef5-5cf286ee313d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.694357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 32/65] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:09.699869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.700164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.700451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.735423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0195-775a-4002-8e27-8e5c7683b306 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:09.744226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:09.745467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:09.749677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:09.780891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:09.781969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.782706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.783442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.784483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:09.784942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:09.787339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58608d06-0a3c-4f44-bc21-194aa02604bd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:09.798343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 34/66] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:09.799980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a2b8-42f6-4c32-b8c3-096c69303eef None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.800456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a2b8-42f6-4c32-b8c3-096c69303eef None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.800915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a2b8-42f6-4c32-b8c3-096c69303eef None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.803582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 35/67] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:09.808795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:09.810072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:09.810561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:09.835330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:09.837625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:09.841555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 33/68] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:09.842872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:09.857628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:09.932750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 36/69] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:09.934318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:09.969347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff48710-0121-4963-ab71-0ff7391c4131 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.015979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.017295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.017583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.017863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.030393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.034133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.035695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.042909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 34/70] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:09 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.043790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.046972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110fb54-f9d0-4bc3-b8ea-67d867eaf066 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.084535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86146b42-1e0e-471c-8119-7e8049cfaf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.085339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86146b42-1e0e-471c-8119-7e8049cfaf0a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.085627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86146b42-1e0e-471c-8119-7e8049cfaf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.085907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86146b42-1e0e-471c-8119-7e8049cfaf0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.097373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86146b42-1e0e-471c-8119-7e8049cfaf0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-86146b42-1e0e-471c-8119-7e8049cfaf0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:10.106914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:10.107997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:10.108968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 37/71] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:10.109659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727ea64-a6e2-4a4d-8b54-ef3aeef4e45d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.109917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727ea64-a6e2-4a4d-8b54-ef3aeef4e45d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.110195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727ea64-a6e2-4a4d-8b54-ef3aeef4e45d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.115367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 38/72] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:10.116329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727ea64-a6e2-4a4d-8b54-ef3aeef4e45d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.134700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.161504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbf524-f0ef-408a-86d1-e984051a6e35 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.198295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.199133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.199521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.199786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.210634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:10.218882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:10.219772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:10.220537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 35/73] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:10.221168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.239982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727ea64-a6e2-4a4d-8b54-ef3aeef4e45d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.273604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59472d-4c7b-4f17-a24f-6fd43dd1a462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.274262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59472d-4c7b-4f17-a24f-6fd43dd1a462 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.274544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59472d-4c7b-4f17-a24f-6fd43dd1a462 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.274821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59472d-4c7b-4f17-a24f-6fd43dd1a462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.279425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.325296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a1eed-10e4-4a6e-b7ad-5fab1e654bc4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.347255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59472d-4c7b-4f17-a24f-6fd43dd1a462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.349337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59472d-4c7b-4f17-a24f-6fd43dd1a462 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.359482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.360158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.360449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.360738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.361101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.361496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:10.361813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.363360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ff9ff-c86d-47d8-807e-ac186aee84fc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.371900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 36/74] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.373233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.373529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.373813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.390408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:10.391666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:10.457745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 39/75] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:10.459370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83e36-dbd1-49d8-aac2-dab8629b01a0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.459665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83e36-dbd1-49d8-aac2-dab8629b01a0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.459934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83e36-dbd1-49d8-aac2-dab8629b01a0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.461281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 40/76] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:10.462003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83e36-dbd1-49d8-aac2-dab8629b01a0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.474304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 37/77] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:10.475597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.528920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.578644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83e36-dbd1-49d8-aac2-dab8629b01a0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.585802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02395-2766-4278-942e-ccd04a10e703 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.590220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.613660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.614996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.615316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.615616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.616361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:10.616671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.618084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62226c-2b56-442c-9804-29ef65f40dc6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.620484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.621370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.621655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.621935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.623174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.624647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.626493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 38/79] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.626594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 41/79] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.627191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.627626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.627964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.628194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.644541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:10.646289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:10.715657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 42/80] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:10.716519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.747424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208e7f-0e92-4e38-9304-edee6e88cc4a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.768634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.795417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.796771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.797235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.797697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.799653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.802017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.804729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 39/81] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.805910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.839022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19496e-cc8e-4429-92fe-c7bd3b24a469 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.841254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:10.883755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.885062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.885548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.886032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.887920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.890483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.898628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 43/82] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.898628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:10.925355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a867b-d869-47b3-bf20-f382f000ad46 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:10.961456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:10.962331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:10.962621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:10.962921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:10.964157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:10.965616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:10.967294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 40/83] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:10.968137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.016285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b5c25-4e56-4e42-8825-c1d11a79a253 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.041110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.065947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.066657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.066948 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.067300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.093807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0824ff-759a-4f23-974d-64a6f58a8920 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.115538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.130840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.131675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.131965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.132248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:11.149551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.151682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:11.152623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:11.153430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:11.154204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 41/84] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:11.155002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.257919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 44/85] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:11.258688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.259525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a90ac-1f8e-4ff1-b65d-cefe67355099 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.276282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.298088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.298759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.299039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.299348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.300181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=a04367f4bff64bab991a70202530d2cc) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:11.300497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.301992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.307986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 42/86] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/role_assignments?user.id=a04367f4bff64bab991a70202530d2cc&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=e59097a97ec144ff8463625775da1081 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:11.308785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.339791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.383302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33ee1a-b8ef-4ed9-900d-05368e366772 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.438009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0db6f7-f361-436f-b1ab-2c4c21261155 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.445059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.446121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.446544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.446987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.448223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.450676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.478609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.479296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.479584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.479909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.491399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 45/87] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:11 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:11.491399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.569607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.610142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.612285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.664059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288a7b5-5bb7-47ba-9362-624126e14d61 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.701710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.724142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.725308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.725738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.726151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.727563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:11.727927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.729784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.740810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 46/88] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:11.741697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.781726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 43/89] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:11.782594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.891295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623fb06-ed74-455a-8c32-ad7dc2567e34 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.908604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:11.939459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de288298-00f4-4934-884d-d3c738d2a540 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:11.939637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.940376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.940709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.941190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.944322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.945809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.947844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 47/90] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:11.948432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:11.982256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:11.983677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.984197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.984672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.986129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:11.987872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:11.992178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 44/91] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:11.993483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:12.011508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:12.062004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8f316-6268-4275-9cd5-82e508f64366 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:12.101268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:12.102258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:12.102557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:12.102842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:12.104103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:12.105707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:12.107913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 48/92] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:12.108726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:12.115094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481ec2b-012d-48c1-add1-0ee7c71bb6f1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:12.154246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:12.155359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:12.155851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:12.156372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:12.157413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=c37a1ed5575d4f4ab7615ea38d6c47b0) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:12.157934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:12.159738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:12.166948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 45/93] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/role_assignments?user.id=c37a1ed5575d4f4ab7615ea38d6c47b0&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:12.168158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:12.183531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:12.225319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377977-5dc9-438c-8140-6fac2dd04668 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:12.263567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:12.270148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:12.275187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:12.275864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:12.276375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:12.309433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfea88c-7df8-4e24-b6ce-1a999e1a251b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:12.358587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:12.359358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:12.359664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:12.359955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:12.361251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] RBAC: Adding query filter params (role.id=852d973419ee442c80da4214c3df867b, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=a04367f4bff64bab991a70202530d2cc) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:12.361578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:12.362415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:12.363151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:12.365872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:12.372474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 46/94] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:12 2022] GET /identity/v3/role_assignments?user.id=a04367f4bff64bab991a70202530d2cc&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=852d973419ee442c80da4214c3df867b => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:12.496629 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 49/95] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:12.502019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:12.600083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:12.633867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28cd23-a66d-4047-bcd1-2d6344f34a29 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:12.680725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07f42-07ff-4ac6-b437-74c3af788b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:12.681616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07f42-07ff-4ac6-b437-74c3af788b0e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:12.681908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07f42-07ff-4ac6-b437-74c3af788b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:12.682192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07f42-07ff-4ac6-b437-74c3af788b0e None admin] PATH_INFO: `/v3/projects/77ec53a35c1845238ffeff61c5058d7c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:12.683434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc07f42-07ff-4ac6-b437-74c3af788b0e None admin] RBAC: Authorizing `identity:get_project(project_id=77ec53a35c1845238ffeff61c5058d7c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:12.684932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc07f42-07ff-4ac6-b437-74c3af788b0e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:12.686949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 47/96] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:12 2022] GET /identity/v3/projects/77ec53a35c1845238ffeff61c5058d7c => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:12.693342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:12.736511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:12.835610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7929f67-3f5a-43bf-89c7-2afeb5675918 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:12.889199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:12.890231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:12.890735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:12.891246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:12.892304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, scope.project.id=77ec53a35c1845238ffeff61c5058d7c, user.id=a04367f4bff64bab991a70202530d2cc) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:12.892848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:12.894612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf17c6c-2822-4dc5-985a-de08aad308dc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:12.905152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 50/97] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:12 2022] GET /identity/v3/role_assignments?user.id=a04367f4bff64bab991a70202530d2cc&scope.project.id=77ec53a35c1845238ffeff61c5058d7c&role.id=e59097a97ec144ff8463625775da1081 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:14.168776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31ea97-a74d-45bc-bc70-08df5b07a178 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:14.169340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31ea97-a74d-45bc-bc70-08df5b07a178 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:14.169340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31ea97-a74d-45bc-bc70-08df5b07a178 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:14.170956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 48/98] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:14.177296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:14.178064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:14.178064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:14.204065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:14.205280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:14.293806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 51/99] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:14.336509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31ea97-a74d-45bc-bc70-08df5b07a178 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:14.463909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31ea97-a74d-45bc-bc70-08df5b07a178 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:14.514679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7fd39-261f-4f82-bf05-a4e9145855e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:14.516633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7fd39-261f-4f82-bf05-a4e9145855e1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:14.517521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7fd39-261f-4f82-bf05-a4e9145855e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:14.518438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7fd39-261f-4f82-bf05-a4e9145855e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:14.541581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7fd39-261f-4f82-bf05-a4e9145855e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1ea7fd39-261f-4f82-bf05-a4e9145855e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:14.554519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:14.555492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:14.556289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:14.559123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 49/100] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:14.568911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:14.592555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:14.608708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a533-e734-4295-8cf0-544868924424 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:14.609018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a533-e734-4295-8cf0-544868924424 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:14.609284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a533-e734-4295-8cf0-544868924424 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:14.610679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 50/101] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:14.615941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:14.616231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:14.616514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:14.627403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:14.648013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:14.649248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:14.753456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61c49e-1cc0-463f-8650-ff79387831e1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:14.836831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:14.837526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:14.837812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:14.838087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:14.838857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:14.839189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:14.840601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:14.849955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 51/102] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:14.854101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 52/103] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:14.859818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:14.889898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.033324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b0c0-823a-4cc1-ab46-8537007b1584 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.066837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:15.077364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc036aab-160f-4cc8-bc48-2c37e9cf80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:15.078718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc036aab-160f-4cc8-bc48-2c37e9cf80df None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.081670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc036aab-160f-4cc8-bc48-2c37e9cf80df None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.082197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc036aab-160f-4cc8-bc48-2c37e9cf80df None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.085644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc036aab-160f-4cc8-bc48-2c37e9cf80df None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:15.088399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc036aab-160f-4cc8-bc48-2c37e9cf80df None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:15.090556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 52/104] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:14 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:15.093487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b16e6-ba67-4843-8896-a22d6d4832e8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.094740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b16e6-ba67-4843-8896-a22d6d4832e8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.095277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b16e6-ba67-4843-8896-a22d6d4832e8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.102737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 53/105] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:15.103845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b16e6-ba67-4843-8896-a22d6d4832e8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.131447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:15.155906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565f16-d3c1-48d4-9b49-90c15517c284 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.229987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73489e01-758b-45ed-a7ee-cca186c40c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:15.232805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489e01-758b-45ed-a7ee-cca186c40c34 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.234125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489e01-758b-45ed-a7ee-cca186c40c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.235301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489e01-758b-45ed-a7ee-cca186c40c34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.237313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73489e01-758b-45ed-a7ee-cca186c40c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:15.241347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73489e01-758b-45ed-a7ee-cca186c40c34 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:15.244642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 53/106] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:15.248822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd9f9c-7451-4bd7-956d-4d9a5d96b2db None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.250985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd9f9c-7451-4bd7-956d-4d9a5d96b2db None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.251583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd9f9c-7451-4bd7-956d-4d9a5d96b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.291583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-33dd9f9c-7451-4bd7-956d-4d9a5d96b2db None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:15.293144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-33dd9f9c-7451-4bd7-956d-4d9a5d96b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:15.315945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b16e6-ba67-4843-8896-a22d6d4832e8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.356092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:15.357382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.357870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.358354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.359786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:15.361449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:15.363429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 54/107] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:15 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:15.364829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.404039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 54/108] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:15.406440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148beb6-c6c9-4f56-b84f-686ea099eefc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.406937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148beb6-c6c9-4f56-b84f-686ea099eefc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.407438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148beb6-c6c9-4f56-b84f-686ea099eefc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.409028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 55/109] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:15.409901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148beb6-c6c9-4f56-b84f-686ea099eefc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.455637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:15.494315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e6344-0e9a-47c2-892e-62a353068c3e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.504305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:15.530058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148beb6-c6c9-4f56-b84f-686ea099eefc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.535494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97c97-1f27-4ab9-8ec6-26f082469bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:15.536523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97c97-1f27-4ab9-8ec6-26f082469bdc None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.536832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97c97-1f27-4ab9-8ec6-26f082469bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.537114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97c97-1f27-4ab9-8ec6-26f082469bdc None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.538274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97c97-1f27-4ab9-8ec6-26f082469bdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:15.540194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97c97-1f27-4ab9-8ec6-26f082469bdc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:15.567621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab216c4-6a14-4cab-9a26-9414c42372dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:15.569048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab216c4-6a14-4cab-9a26-9414c42372dc None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.569576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab216c4-6a14-4cab-9a26-9414c42372dc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.570068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab216c4-6a14-4cab-9a26-9414c42372dc None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a/users/a04367f4bff64bab991a70202530d2cc/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.570502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 55/110] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:18:15 2022] POST /identity/v3/projects => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:15.571839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7f6b-d750-45e5-9832-b56c236a9403 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.572135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7f6b-d750-45e5-9832-b56c236a9403 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.572418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7f6b-d750-45e5-9832-b56c236a9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.577626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab216c4-6a14-4cab-9a26-9414c42372dc None admin] RBAC: Authorizing `identity:create_grant(project_id=3be9f13173874417a10d07a7e522883a, user_id=a04367f4bff64bab991a70202530d2cc, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:15.579337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab216c4-6a14-4cab-9a26-9414c42372dc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:15.594723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6e8b7f6b-d750-45e5-9832-b56c236a9403 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:15.596439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6e8b7f6b-d750-45e5-9832-b56c236a9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:15.599737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 56/111] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:15 2022] PUT /identity/v3/projects/3be9f13173874417a10d07a7e522883a/users/a04367f4bff64bab991a70202530d2cc/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:15.601498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042010ed-9b06-43a0-a1ed-dfe228d37cea None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.601862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042010ed-9b06-43a0-a1ed-dfe228d37cea None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.602151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042010ed-9b06-43a0-a1ed-dfe228d37cea None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.607137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 57/112] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:15.610351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042010ed-9b06-43a0-a1ed-dfe228d37cea None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.695423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 56/113] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:15.696685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.696989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.697272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.716769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:15.718969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:15.745571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042010ed-9b06-43a0-a1ed-dfe228d37cea None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.785485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:15.797387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:15.798518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:15.798813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:15.799123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:15.811220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:15.816222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 57/114] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:15.823197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:15.824097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:15.830085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:15.831228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:15.832167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:15.841447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 58/115] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:15.842510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:15.977232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1da3ea-e7e7-4050-9504-c26df3b927bb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:15.987102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91845bdd-34f6-4ba5-b22e-57b62b3816c5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.017600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.018766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.019374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.019915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.022865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.023739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.023857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.024118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.025139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:16.025448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.026862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.033548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:16.036267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 59/116] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:16.037104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:16.044166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:16.045268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:16.046134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:16.048315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 58/117] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:16.049287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.102722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.124403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.170693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee5ada-db74-4453-a5a9-b11b1705c618 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.179328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652529ee-28a8-4472-a07d-cf8c3315d333 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.212487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.214333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.214804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.215409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] PATH_INFO: `/v3/users/e667e15be5c14f00b2d8d9c936c7fc3a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.220203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc86146-8e6d-4033-b673-c43cade5f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.220880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc86146-8e6d-4033-b673-c43cade5f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.221176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc86146-8e6d-4033-b673-c43cade5f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.221456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc86146-8e6d-4033-b673-c43cade5f2c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.233610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e667e15be5c14f00b2d8d9c936c7fc3a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.233817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc86146-8e6d-4033-b673-c43cade5f2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:16.235942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.238373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 59/118] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/users/e667e15be5c14f00b2d8d9c936c7fc3a => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:16.239620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2cc86146-8e6d-4033-b673-c43cade5f2c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:16.242253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:16.243138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:16.243911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 60/119] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:16.245158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d5412-eb24-4bca-b981-64116c84fb8f None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.250010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d5412-eb24-4bca-b981-64116c84fb8f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.250326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d5412-eb24-4bca-b981-64116c84fb8f None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.252166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 61/120] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:16.253191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d5412-eb24-4bca-b981-64116c84fb8f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.361059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.366824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0baf5b-2341-413d-be0f-60f43d51c9a3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.378435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d5412-eb24-4bca-b981-64116c84fb8f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.402492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.410286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.411296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.411794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.412291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.413473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:16.413971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.416018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.418758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.420912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.421200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.421478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.427530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 60/121] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:16.428658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.528240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.530019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.552653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f765-a471-481b-82e7-240fe2f39f02 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.590712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.591808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.592271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.592761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.593833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:16.594380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.595998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad8c02-6226-416f-9eae-246703fcf703 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.606045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 61/122] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:16.607579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.608051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.608570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.627564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:16.629048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:16.648629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.652683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 62/123] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:16.653598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.681493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.752600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 62/124] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:16.755342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.794860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724eb3-4ec1-4a94-b22f-3d741eb967fa None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.840010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.840883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.841168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.841444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.842648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.844163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.851805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 63/125] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:16.852850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.883136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46e373-c262-4147-b535-135caabd3f9f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:16.921816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:16.922737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:16.923026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:16.923366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:16.924639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:16.927072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:16.928902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 63/126] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:16.929522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:16.956922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.971510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:16.986536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd14e1-0e70-4801-a03f-2849b2d21617 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.022787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.023696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.023984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.024318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.026171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.027677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.029500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 64/127] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:17.030340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.067996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3377b4-a0c9-4f22-8b2e-cb05685457b4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.114229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.115043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.115385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.115656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.128058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:17.137221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:17.138303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:17.139264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 64/128] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:17.140126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.175489 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a2e58-57ea-4245-a0f6-e26bb6217daf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.219680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.221511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.222199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.223838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.227916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.229934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.232503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 65/129] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:17 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:17.233914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.248390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:17.268041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:17.271672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932422-5021-4bb9-a79e-32b68eea5837 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.317264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.318542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.318787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.319082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.320557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.322195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.324191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 65/130] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:17 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:17.324985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.371103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db43c4-5901-464d-8295-75b03c88e5f1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.413229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.414295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.414795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.415429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.416545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] RBAC: Adding query filter params (role.id=f2d22e4432674f50ae6d73b8a5af344b, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=e667e15be5c14f00b2d8d9c936c7fc3a) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:17.417104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.418960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.426796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 66/131] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:17 2022] GET /identity/v3/role_assignments?user.id=e667e15be5c14f00b2d8d9c936c7fc3a&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=f2d22e4432674f50ae6d73b8a5af344b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:17.428257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.460931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffe56-77f9-4115-8788-b78c5c4e65b3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.533524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.534168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.534459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.534734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.536061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:17.536372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.547611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:17.549432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.558497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855b8e-fbd2-48d3-85d2-3b1fbc5cf7e8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.568968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 66/132] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:17.569890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.593346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:17.609792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.611172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.611673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.612176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a/users/a04367f4bff64bab991a70202530d2cc/roles/852d973419ee442c80da4214c3df867b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.615456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be9f13173874417a10d07a7e522883a, user_id=a04367f4bff64bab991a70202530d2cc, role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.617300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.639556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 67/133] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:17 2022] PUT /identity/v3/projects/3be9f13173874417a10d07a7e522883a/users/a04367f4bff64bab991a70202530d2cc/roles/852d973419ee442c80da4214c3df867b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:17.648354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:17.708144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f451b-b60d-491b-9bca-56320b936148 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.748752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.749741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.750032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.750312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a/users/c37a1ed5575d4f4ab7615ea38d6c47b0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.752776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be9f13173874417a10d07a7e522883a, user_id=c37a1ed5575d4f4ab7615ea38d6c47b0, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.763083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.771975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 67/134] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:17 2022] PUT /identity/v3/projects/3be9f13173874417a10d07a7e522883a/users/c37a1ed5575d4f4ab7615ea38d6c47b0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:17.837929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f465b1-de6a-49ec-9818-252fb9810556 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:17.900466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:17.901715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:17.902224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:17.902703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:17.904290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:17.906198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:17.908133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 68/135] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:17 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:17.914890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:18.010637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:18.046985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde23760-7a6b-42b3-9408-f1d6ea7a98d0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:18.086165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e3d6c-7d91-43c4-a200-889bad1086f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:18.087084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3d6c-7d91-43c4-a200-889bad1086f1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.087378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3d6c-7d91-43c4-a200-889bad1086f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.087659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e3d6c-7d91-43c4-a200-889bad1086f1 None admin] PATH_INFO: `/v3/projects/77ec53a35c1845238ffeff61c5058d7c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.089024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3d6c-7d91-43c4-a200-889bad1086f1 None admin] RBAC: Authorizing `identity:get_project(project_id=77ec53a35c1845238ffeff61c5058d7c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:18.090565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e3d6c-7d91-43c4-a200-889bad1086f1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:18.092587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 68/136] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:17 2022] GET /identity/v3/projects/77ec53a35c1845238ffeff61c5058d7c => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:18.099018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:18.140384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:18.247337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcdf41-7732-495e-bba3-f863ec1ecf79 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:18.293417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:18.294710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.295226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.295749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] PATH_INFO: `/v3/projects/77ec53a35c1845238ffeff61c5058d7c/users/a04367f4bff64bab991a70202530d2cc/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.298073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] RBAC: Authorizing `identity:create_grant(project_id=77ec53a35c1845238ffeff61c5058d7c, user_id=a04367f4bff64bab991a70202530d2cc, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:18.299810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:18.302553 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd6be-171b-4458-8cec-3d3d9fff25a3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.302848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd6be-171b-4458-8cec-3d3d9fff25a3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.303193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd6be-171b-4458-8cec-3d3d9fff25a3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.304842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 69/137] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:18.310800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9727157-8c2a-4280-80f6-24c17533376d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.311211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9727157-8c2a-4280-80f6-24c17533376d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.311508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9727157-8c2a-4280-80f6-24c17533376d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.326815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 69/138] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:18 2022] PUT /identity/v3/projects/77ec53a35c1845238ffeff61c5058d7c/users/a04367f4bff64bab991a70202530d2cc/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:18.333609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b9727157-8c2a-4280-80f6-24c17533376d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:18.334933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b9727157-8c2a-4280-80f6-24c17533376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:18.416945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 70/139] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:18.440679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:18.505714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48aa2-aab6-4736-aefe-f41a7280c15e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.506216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48aa2-aab6-4736-aefe-f41a7280c15e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.506216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48aa2-aab6-4736-aefe-f41a7280c15e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.508436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 71/140] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:18.513945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.514247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.514555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.534942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:18.536323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:18.559470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:18.595703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2dc82-d3de-48a8-8e1b-ca5f55701ff9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:18.650190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:18.666799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 72/141] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 22:18:18.685812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:18.686647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:18.686937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:18.687247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:18.702262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:18.716188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:18.724474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:18.725379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:18.726146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:18.731557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 70/142] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:18.737009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:18.911857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca81d26-0c2d-41fd-82c1-955f35d15a39 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:18.952526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8a119-a328-4060-9f4e-e828da709d67 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:19.003886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:19.005447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.005737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.006017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.008684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:19.010163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:19.022931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 73/143] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:19.031273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:19.037938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:19.038625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.038916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.039238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.040015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:19.040330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:19.041724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:19.050874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 71/144] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:19.052176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:19.057660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:19.165660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:19.173501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6f38-dcae-44e8-80bf-92439690b590 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:19.219839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:19.220722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.221017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.221304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.222587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:19.224784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:19.226832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 72/145] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:19 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:19.232281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:19.239234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3fde1-28ef-4c21-9e99-1121175edd48 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:19.341899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:19.342595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.342884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.343194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.343946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:19.345428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:19.357606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:19.416871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 74/146] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:19 2022] POST /identity/v3/users => generated 283 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:18:19.436545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87019c82-2bc5-4294-ac54-7dc20a10009b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:19.490884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:19.491596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.491887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.492169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.586625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:19.592547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:19.685029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:19.724730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 73/147] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:18:19.730423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:19.852997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b20ed2-fdb2-4717-b5b8-31fab5f69d2e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:19.879156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:19.892754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272b01e-3c57-4e1d-a091-7bb9e9695214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:19.893672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272b01e-3c57-4e1d-a091-7bb9e9695214 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.899612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272b01e-3c57-4e1d-a091-7bb9e9695214 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.900168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272b01e-3c57-4e1d-a091-7bb9e9695214 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.901689 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272b01e-3c57-4e1d-a091-7bb9e9695214 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:19.903288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272b01e-3c57-4e1d-a091-7bb9e9695214 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:19.905154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 75/148] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:19 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:19.911786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:20.020693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bda84-b591-46bb-bc4f-255a2eaefdf4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:20.057166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:20.057840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.058129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.058410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.059358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=a04367f4bff64bab991a70202530d2cc) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:20.059683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:20.061219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafed20-788c-43c0-9a27-c08ecd426bdd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:20.067946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 74/149] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:19 2022] GET /identity/v3/role_assignments?user.id=a04367f4bff64bab991a70202530d2cc&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=e59097a97ec144ff8463625775da1081 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:20.814536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe536b13-11d4-4f04-a073-8c92522a52cc None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.815043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe536b13-11d4-4f04-a073-8c92522a52cc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.815189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe536b13-11d4-4f04-a073-8c92522a52cc None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.817116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 76/150] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:20.823140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da618138-c01b-4c74-b086-67549c074c73 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.823601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da618138-c01b-4c74-b086-67549c074c73 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.823789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da618138-c01b-4c74-b086-67549c074c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.844301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-da618138-c01b-4c74-b086-67549c074c73 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:20.845517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-da618138-c01b-4c74-b086-67549c074c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:20.858507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984596e3-a7c8-468e-9d8b-76b7cbc095fb None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.858820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984596e3-a7c8-468e-9d8b-76b7cbc095fb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.859096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984596e3-a7c8-468e-9d8b-76b7cbc095fb None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.862781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 77/151] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:20.868453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31901-cde3-4401-b639-674392e02b4d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.868740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31901-cde3-4401-b639-674392e02b4d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.869023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31901-cde3-4401-b639-674392e02b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.900876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-88e31901-cde3-4401-b639-674392e02b4d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:20.902128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-88e31901-cde3-4401-b639-674392e02b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:20.910974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:20.991826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 75/152] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:21.017912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da618138-c01b-4c74-b086-67549c074c73 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:21.080127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 78/153] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:21.133998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31901-cde3-4401-b639-674392e02b4d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:21.226167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da618138-c01b-4c74-b086-67549c074c73 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:21.280095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebddbd-4095-4562-a626-7482f593e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:21.280910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebddbd-4095-4562-a626-7482f593e5ed None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.281193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebddbd-4095-4562-a626-7482f593e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.281469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebddbd-4095-4562-a626-7482f593e5ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.294175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebddbd-4095-4562-a626-7482f593e5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:21.294686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31901-cde3-4401-b639-674392e02b4d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-85ebddbd-4095-4562-a626-7482f593e5ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:21.302890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:21.304033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:21.304909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 76/154] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:21.306455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8d179-a4b4-4d8c-a596-78b77c439f81 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.306912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8d179-a4b4-4d8c-a596-78b77c439f81 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.307354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8d179-a4b4-4d8c-a596-78b77c439f81 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.311597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 77/155] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:21.313273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7eed6-bac7-4259-9651-489f2867a226 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.313804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7eed6-bac7-4259-9651-489f2867a226 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.314258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7eed6-bac7-4259-9651-489f2867a226 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.317928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 78/156] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:21.319556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7eed6-bac7-4259-9651-489f2867a226 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:21.325693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:21.348003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e1a99-c91e-4d24-ad56-03caebdaa7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:21.350065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e1a99-c91e-4d24-ad56-03caebdaa7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.350609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e1a99-c91e-4d24-ad56-03caebdaa7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.350895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e1a99-c91e-4d24-ad56-03caebdaa7c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.356611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:21.365879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e1a99-c91e-4d24-ad56-03caebdaa7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f83e1a99-c91e-4d24-ad56-03caebdaa7c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:21.379529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:21.381088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:21.382176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 79/157] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:21.389655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af276503-db4b-40a4-95c8-74832d1f0d1d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.389938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af276503-db4b-40a4-95c8-74832d1f0d1d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.390224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af276503-db4b-40a4-95c8-74832d1f0d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.409728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-af276503-db4b-40a4-95c8-74832d1f0d1d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:21.410938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-af276503-db4b-40a4-95c8-74832d1f0d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:21.489000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 80/158] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:21.490348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.490646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.490924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.512808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:21.512808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:21.565412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7eed6-bac7-4259-9651-489f2867a226 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:21.595083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 81/159] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:21.595083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:21.637105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:21.660137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:21.661631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.662153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.662252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.663693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:21.664174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:21.665601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:21.687138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 79/160] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:21.693280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:21.768721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e63a-5095-4bf2-a893-e2ad2e70eab4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:21.820345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:21.821137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.821351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.821583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:21.822357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:21.822664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:21.824123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:21.848517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 82/161] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:21.849646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:21.907298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857d36-01b9-4b3b-8031-c0451dae1df6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:21.911416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:21.980992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:21.982290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:21.982783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:21.983286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.008124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:22.017027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:22.017943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:22.018703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:22.025740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 80/162] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:22.028101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.059658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8c38-237d-4856-97b6-9d28694ed48f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.061822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:22.126004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.127392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.129659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.130979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.155960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:22.163778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:22.164755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 83/163] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:22.165891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.203918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6a3df-4ac8-4872-bfc2-23bf77655635 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.248682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.250048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.250476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.251002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] PATH_INFO: `/v3/users/e667e15be5c14f00b2d8d9c936c7fc3a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.252551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] RBAC: Authorizing `identity:get_user(user_id=e667e15be5c14f00b2d8d9c936c7fc3a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:22.255714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:22.257898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 81/164] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/users/e667e15be5c14f00b2d8d9c936c7fc3a => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:22.258902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.279248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:22.394919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c64e4-cc92-460e-9c68-8dd3a3476564 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.434766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.435688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42183f60-089a-4aba-8101-5c08419d8af6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.436153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.436583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.437125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.438495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:22.440212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:22.442776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 84/165] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:22.443823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.485236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:22.511935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.512659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.512979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.513265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.514045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:22.514357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:22.515816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:22.525685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 82/166] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:22.526743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.571391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407232f-975a-4548-8dd8-126dcbb4a034 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.612505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.613604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.614142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.614627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.615660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:22.616178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:22.617813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:22.628036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 85/167] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:22.629490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.644062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:22.705471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584258-59ed-4a51-8f34-80a6d5ae64e0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.744243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:22.756739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b465c-5bea-4c8e-99be-59c9b471c248 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:22.777160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.778039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.778326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.778604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.783794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:22.785317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:22.788956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 83/168] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:22.788956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:22.821699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:22.822354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:22.822582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:22.822826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:22.933038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:22.934651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.033278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb9b43-de3b-4b58-b54d-d96dc9f822c9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.034096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.046633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 86/169] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:23.047514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.085991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.086785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.087088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.087368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] PATH_INFO: `/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.088586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] RBAC: Authorizing `identity:get_user(user_id=c37a1ed5575d4f4ab7615ea38d6c47b0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.090046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.092213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 84/170] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:22 2022] GET /identity/v3/users/c37a1ed5575d4f4ab7615ea38d6c47b0 => generated 247 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:23.093148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.105398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.162361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd4d0-f08d-4103-af64-2d0316921400 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.205448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.206331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.206615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.207483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] PATH_INFO: `/v3/users/a04367f4bff64bab991a70202530d2cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.208421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa5f2b-a293-451f-8148-cc2517e911fc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.209360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] RBAC: Authorizing `identity:get_user(user_id=a04367f4bff64bab991a70202530d2cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.211101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.212822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 87/171] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/users/a04367f4bff64bab991a70202530d2cc => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:23.213678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.249503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a19d6e-a2c9-441f-9ec4-f4aebc98d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.250545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19d6e-a2c9-441f-9ec4-f4aebc98d225 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.250963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19d6e-a2c9-441f-9ec4-f4aebc98d225 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.251268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19d6e-a2c9-441f-9ec4-f4aebc98d225 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a/users/e667e15be5c14f00b2d8d9c936c7fc3a/roles/f2d22e4432674f50ae6d73b8a5af344b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.255253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a19d6e-a2c9-441f-9ec4-f4aebc98d225 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be9f13173874417a10d07a7e522883a, user_id=e667e15be5c14f00b2d8d9c936c7fc3a, role_id=f2d22e4432674f50ae6d73b8a5af344b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.256722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a19d6e-a2c9-441f-9ec4-f4aebc98d225 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.275005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 85/172] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:23 2022] PUT /identity/v3/projects/3be9f13173874417a10d07a7e522883a/users/e667e15be5c14f00b2d8d9c936c7fc3a/roles/f2d22e4432674f50ae6d73b8a5af344b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:23.276529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5887b-d9a5-41c3-a32c-06268682525e None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.276817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5887b-d9a5-41c3-a32c-06268682525e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.277092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5887b-d9a5-41c3-a32c-06268682525e None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.283272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 86/173] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:23.284323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5887b-d9a5-41c3-a32c-06268682525e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.293561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.381449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.392307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1488b-b012-4c86-b475-ac3d614ed09c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.437752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5887b-d9a5-41c3-a32c-06268682525e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.487200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff912e-eb39-45cc-ae80-eee237118e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.489015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff912e-eb39-45cc-ae80-eee237118e22 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.489306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff912e-eb39-45cc-ae80-eee237118e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.489644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff912e-eb39-45cc-ae80-eee237118e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.503957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.504869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.505154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.505435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.506782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.508317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.510127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 88/174] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:23.510945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.592025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff912e-eb39-45cc-ae80-eee237118e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.593545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff912e-eb39-45cc-ae80-eee237118e22 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.684028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.701748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82362b-09d0-4438-8595-2e4864808446 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.731628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 87/175] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:23.733815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.734318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.734819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.743088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.751900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067ea29-062a-4e03-97ea-4adac548c0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.752557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067ea29-062a-4e03-97ea-4adac548c0cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.752837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067ea29-062a-4e03-97ea-4adac548c0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.753112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067ea29-062a-4e03-97ea-4adac548c0cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.764440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:23.766664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:23.850208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 88/176] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:23.851244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067ea29-062a-4e03-97ea-4adac548c0cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.852111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.854151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067ea29-062a-4e03-97ea-4adac548c0cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.965648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:23.973094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 89/177] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:23.973094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bbe88-bbd9-47be-b7d7-a15d04499c92 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.973374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd83835-bc1f-4d79-b791-503818622f4a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.973482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bbe88-bbd9-47be-b7d7-a15d04499c92 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.973482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bbe88-bbd9-47be-b7d7-a15d04499c92 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.973482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 90/178] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:23.973482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1a97-60b1-4a0e-83cc-7f3fdf0385d5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.973482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1a97-60b1-4a0e-83cc-7f3fdf0385d5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.973771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1a97-60b1-4a0e-83cc-7f3fdf0385d5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.975724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 91/179] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:23.979164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f1a97-60b1-4a0e-83cc-7f3fdf0385d5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.011711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.017677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.018352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.018635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.018915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.019914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] RBAC: Adding query filter params (role.id=852d973419ee442c80da4214c3df867b, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=a04367f4bff64bab991a70202530d2cc) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:24.020248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.021949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.039311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 89/180] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/role_assignments?user.id=a04367f4bff64bab991a70202530d2cc&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=852d973419ee442c80da4214c3df867b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.039311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.142890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f1a97-60b1-4a0e-83cc-7f3fdf0385d5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.161053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be1aa8-59c3-4b64-b892-203a8f605565 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.198203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.198566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.198854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.199160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.200675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.202837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.204445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 92/181] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.206380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.209044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3762b-e988-44bb-8e09-98f1d13a8fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.210204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3762b-e988-44bb-8e09-98f1d13a8fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.210654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3762b-e988-44bb-8e09-98f1d13a8fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.211153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3762b-e988-44bb-8e09-98f1d13a8fe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.225774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3762b-e988-44bb-8e09-98f1d13a8fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e2d3762b-e988-44bb-8e09-98f1d13a8fe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:24.235255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:24.236320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:24.237215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:24.239757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 90/182] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:24.245880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef52330-7b25-4356-969a-825e65f5559c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.245880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef52330-7b25-4356-969a-825e65f5559c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.245880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef52330-7b25-4356-969a-825e65f5559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.265135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6ef52330-7b25-4356-969a-825e65f5559c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:24.267088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6ef52330-7b25-4356-969a-825e65f5559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:24.293518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.337652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.345241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f67567-0df2-495d-8794-ea8e4805f733 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.358647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 91/183] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:24.360013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.360303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.360576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.377251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:24.378543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:24.390819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.391980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.392269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.392545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] PATH_INFO: `/v3/projects/77ec53a35c1845238ffeff61c5058d7c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.393742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] RBAC: Authorizing `identity:get_project(project_id=77ec53a35c1845238ffeff61c5058d7c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.395945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.397808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 93/184] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/projects/77ec53a35c1845238ffeff61c5058d7c => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.398507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.454619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 92/185] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:24.455782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.514611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d2265-93eb-4e0b-a7cd-03058989c622 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.542880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.554847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.555550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.555835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.556113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.557084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=c37a1ed5575d4f4ab7615ea38d6c47b0) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:24.557519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.559017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.563149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e05fd-5b3d-4e8e-b7b8-47098571fefa None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.565771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 94/186] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/role_assignments?user.id=c37a1ed5575d4f4ab7615ea38d6c47b0&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.566851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.613566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.620223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.620992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.622152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.622677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.623575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:24.623894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.625484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.637436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 93/187] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.638178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.719815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7ea9f-577f-4071-872f-d35dd5be7dd2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.762635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.763840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.764354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.764931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.766358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.768235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.771091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7d547-aae4-4d55-bc99-0e3dad547144 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.772092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 95/188] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.773562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.818921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.819618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.819903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.820174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.821044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, scope.project.id=77ec53a35c1845238ffeff61c5058d7c, user.id=a04367f4bff64bab991a70202530d2cc) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:24.821357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:24.822923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:24.829719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 94/189] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/role_assignments?user.id=a04367f4bff64bab991a70202530d2cc&scope.project.id=77ec53a35c1845238ffeff61c5058d7c&role.id=e59097a97ec144ff8463625775da1081 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:24.830575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:24.843158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.896617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e74aa1-23ee-4c1b-8a47-8dcd6d1982c5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.905013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:24.946757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ba7c5-63c1-499e-8c1e-b9061c2b6364 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:24.960572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.961899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.962455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.962943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.977286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:24.984025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:24.984912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:24.985194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:24.985470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] PATH_INFO: `/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:24.989336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:24.990684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:24.991678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:24.995578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 96/190] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:24.996808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.004296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] RBAC: Authorizing `identity:get_user(user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.005807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.007667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 95/191] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:24 2022] GET /identity/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:25.008461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.118555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.143434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7940-66c2-4fcd-b0ae-71eacb441817 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.162873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995446f-abee-49da-88a1-e93c54123a5a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.182595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.183304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.183595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.183923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.184650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.186181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.206781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.208243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.208534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.208808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.209511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 97/192] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:24 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:25.210351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.215231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:25.216038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.217891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.229952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.233562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 96/193] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:25.239836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.349469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc637b-cc76-4d18-bb02-e8d95aa692d4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.387739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67cbc3-03b5-42bd-9ceb-2613c28ed7e4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.400912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.402236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.402518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.402790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.426963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.443747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.446368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.446656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.446937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] PATH_INFO: `/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.450110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.451819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.453551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 97/194] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:25.459469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.500660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.502153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.524241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.604961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7e3d8-f0d1-4f93-bfa5-be5aa9b6cc89 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.639792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 98/195] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:25.648412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.653722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94200c1f-83fe-4ad8-a084-d16879762ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.654404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94200c1f-83fe-4ad8-a084-d16879762ecd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.654692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94200c1f-83fe-4ad8-a084-d16879762ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.655083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94200c1f-83fe-4ad8-a084-d16879762ecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.733228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.741733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94200c1f-83fe-4ad8-a084-d16879762ecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.743584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94200c1f-83fe-4ad8-a084-d16879762ecd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.818550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db7cf-0412-417a-9dd2-69c0bec1f59b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.839468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.862361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 98/196] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:18:25.863823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851f32a-736b-46bf-9c6a-80011ac326af None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.864100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851f32a-736b-46bf-9c6a-80011ac326af None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.864374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851f32a-736b-46bf-9c6a-80011ac326af None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.866418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 99/197] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:25.871882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851f32a-736b-46bf-9c6a-80011ac326af None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.874923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523524f6-87af-4363-af41-d46167f37bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.877635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523524f6-87af-4363-af41-d46167f37bcc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.878203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523524f6-87af-4363-af41-d46167f37bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.879242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523524f6-87af-4363-af41-d46167f37bcc None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.880848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523524f6-87af-4363-af41-d46167f37bcc None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:25.882560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523524f6-87af-4363-af41-d46167f37bcc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:25.885325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 99/198] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:25.886818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.887517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.888036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.912837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:25.914812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:25.995195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 100/199] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:25.996536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.029191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.044789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851f32a-736b-46bf-9c6a-80011ac326af None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.109821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.110886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.111445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.111716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.113916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.115696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.117605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 100/200] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:26.123480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.158702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf0e57-0d46-43b6-af35-c1f2f8093ba2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.180285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.225345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.226476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.226982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.227522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.229002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:26.229577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.232572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.246196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e7cd1-04a0-4550-9e3f-751bed417f10 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.251454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 101/201] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/role_assignments?user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:26.252723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.291031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.292443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.293067 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.293514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.333394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:26.353582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:26.354908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:26.355864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:26.361925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 101/202] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:26.372404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.429763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.480979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c754a-6081-46e6-8abf-bb0e60748a52 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.572491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.573157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.573446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.573727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.574572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] RBAC: Adding query filter params (role.id=852d973419ee442c80da4214c3df867b, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:26.574887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.576012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00665885-84ac-4793-8fda-57c4e95590e2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.576406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.593202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 102/203] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/role_assignments?user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=852d973419ee442c80da4214c3df867b => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:26.619939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.621158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.621748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.622297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.623413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:26.624004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.625789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.637630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 102/204] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:26.643635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.730025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.841967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004d92-7a84-4a23-914b-a08d629228e4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.904858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.905801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.906085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.906360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] PATH_INFO: `/v3/users/e667e15be5c14f00b2d8d9c936c7fc3a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.908263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] RBAC: Authorizing `identity:get_user(user_id=e667e15be5c14f00b2d8d9c936c7fc3a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.910715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.913095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 103/205] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/users/e667e15be5c14f00b2d8d9c936c7fc3a => generated 289 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:26.919135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:27.095169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:27.111625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7191c-0a3f-488f-8015-2c47935b8c99 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:27.192750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:27.199080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:27.199080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:27.199080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:27.329528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:27.331264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:27.435104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:27.463909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 103/206] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:27.472207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:27.685503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d3da-bd4c-425c-b7f7-c93c91e04485 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:27.695765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:27.734446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123a455-6d3d-4cc3-a2d7-f82112651bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:27.735363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123a455-6d3d-4cc3-a2d7-f82112651bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:27.735988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123a455-6d3d-4cc3-a2d7-f82112651bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:27.736284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123a455-6d3d-4cc3-a2d7-f82112651bb8 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:27.737543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123a455-6d3d-4cc3-a2d7-f82112651bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:27.740187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123a455-6d3d-4cc3-a2d7-f82112651bb8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:27.742047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 104/207] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:27 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:27.748210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:27.859126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44831c7-8621-4dc1-b125-766546d8df1c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:27.897371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:27.898046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:27.898330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:27.898607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:27.899750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] RBAC: Adding query filter params (role.id=f2d22e4432674f50ae6d73b8a5af344b, scope.project.id=3be9f13173874417a10d07a7e522883a, user.id=e667e15be5c14f00b2d8d9c936c7fc3a) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:27.900064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:27.901554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99161c9-57b1-4ae4-8b5c-9ce0ccaa7bef None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:27.908263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 104/208] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:27 2022] GET /identity/v3/role_assignments?user.id=e667e15be5c14f00b2d8d9c936c7fc3a&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=f2d22e4432674f50ae6d73b8a5af344b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:28.237258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929ab13-02ed-46ad-9d4f-2bf3905df233 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.237784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929ab13-02ed-46ad-9d4f-2bf3905df233 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.237894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929ab13-02ed-46ad-9d4f-2bf3905df233 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.240881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 105/209] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:28.248687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.248687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.248687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.266774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:28.268143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda178-c331-4b0c-bfb8-562210f0adab None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.268431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda178-c331-4b0c-bfb8-562210f0adab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.268710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda178-c331-4b0c-bfb8-562210f0adab None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.269027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:28.270931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 106/210] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:28.277966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.278253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.278533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.302600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:28.303997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:28.358312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:28.360564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 105/211] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:28.394027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:28.398845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 107/212] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:28.424975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:28.514112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729c67-2670-4d0a-977d-8405876bc8d6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:28.551100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd0af-db07-4052-9281-aa856192b31d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:28.554126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:28.558773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:28.559932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.561435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.561968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.574579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:28.583990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:28.584920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:28.585719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:28.593073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:28.593073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.593073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.593073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.593461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:28.593775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 106/213] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:28.594148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:28.595342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:28.596263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 108/214] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:28.602914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:28.645731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:28.712050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e5ac-cff2-47ef-8e55-222b6c22ec07 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:28.716884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce9f0-d77a-421d-ad2e-6a843a66d211 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:28.749204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:28.749897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.750186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.750463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.751227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:28.752724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:28.757876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:28.758854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.759467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.759946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.761042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:28.761651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:28.763400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:28.774077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 107/215] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:28.780580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:28.788495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 109/216] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:28 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:18:28.909108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb970a-1ded-4ff0-90cc-bb9b8b8074a6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:28.912257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:28.951171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d52a3-65aa-4b66-8bab-724b41a1fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:28.952800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d52a3-65aa-4b66-8bab-724b41a1fe09 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:28.953416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d52a3-65aa-4b66-8bab-724b41a1fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:28.953910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d52a3-65aa-4b66-8bab-724b41a1fe09 None admin] PATH_INFO: `/v3/users/d1e7773dac5a488a81f5cb4297d8f2ef` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:28.970476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d52a3-65aa-4b66-8bab-724b41a1fe09 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e7773dac5a488a81f5cb4297d8f2ef)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:28.973413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d52a3-65aa-4b66-8bab-724b41a1fe09 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:28.975525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 108/217] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:28 2022] GET /identity/v3/users/d1e7773dac5a488a81f5cb4297d8f2ef => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:28.985103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:29.030398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:29.122163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90e3b9-0564-44f2-a834-c8943ca4d380 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:29.163975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a4476-cff9-41e6-9217-2fe4aaae15af None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.164260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:29.164424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a4476-cff9-41e6-9217-2fe4aaae15af None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.164558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a4476-cff9-41e6-9217-2fe4aaae15af None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.165102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.165252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.165586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.165917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 109/218] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:29.171435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.171819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.172230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.200087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:29.202625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:29.305194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:29.306631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:29.319398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 110/219] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:29.348320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:29.350287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:29.428449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 110/220] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:29.434257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:29.454159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:29.465964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3215da-7023-4df8-8945-70333d0498ae None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:29.507181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:29.508018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.508317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.508576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.521152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:29.530015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:29.533645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 111/221] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:29.543310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:29.555106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32291ebd-9ea3-497c-b80b-48098ae907b2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:29.583809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:29.587335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:29.588415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.588805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.589241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.590606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:29.592254 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:29.594429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 111/222] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:29 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:29.600587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:29.711014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e869c-f617-405e-af2a-13689186f1f4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:29.716770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9ada6-2c45-4fed-9cc8-c655b2a15807 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:29.744508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:29.745292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.745638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.745977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.746893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=d1e7773dac5a488a81f5cb4297d8f2ef) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:29.747350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:29.748918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:29.755917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 112/223] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:29 2022] GET /identity/v3/role_assignments?user.id=d1e7773dac5a488a81f5cb4297d8f2ef&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=e59097a97ec144ff8463625775da1081 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:29.855091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:29.856357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:29.856875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:29.857380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:29.858902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:29.859508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:29.862173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:29.883189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 112/224] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:29.892294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:29.964324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:30.007569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1328-de38-4ce1-8dd4-c694228eff3b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:30.056396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:30.057293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.057574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.057831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] PATH_INFO: `/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.059303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:30.060837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:30.063859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 113/225] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:29 2022] GET /identity/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:30.068755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:30.091652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:30.186902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60a4d2-408f-4efd-a515-e3ced47410ef None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:30.223871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba83ef-62d9-473c-9b2f-d0b0d4c7e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:30.224764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba83ef-62d9-473c-9b2f-d0b0d4c7e794 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.225044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba83ef-62d9-473c-9b2f-d0b0d4c7e794 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.225315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba83ef-62d9-473c-9b2f-d0b0d4c7e794 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.226683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba83ef-62d9-473c-9b2f-d0b0d4c7e794 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:30.228200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba83ef-62d9-473c-9b2f-d0b0d4c7e794 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:30.230229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 113/226] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:30 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:30.236204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:30.279744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e21de-cc8c-45bc-8a2b-b792b9e54b03 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.280160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e21de-cc8c-45bc-8a2b-b792b9e54b03 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.280431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e21de-cc8c-45bc-8a2b-b792b9e54b03 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.281679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 114/227] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:30.290146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.291102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.291888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.320927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:30.324839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:30.414564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 115/228] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:30.451360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab65464-dc00-4329-a03f-9d935fbb4979 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:30.464134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:30.484187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:30.533283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:30.545671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:30.546716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.547005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.555489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75/users/1ac6c9a3c7ae4e34b90fdb3b3111272a/roles/852d973419ee442c80da4214c3df867b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.558004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff45f877814dada97a096ebf743f75, user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a, role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:30.559556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:30.578084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804f737-09b6-4c71-a27a-072544d0dbcb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:30.602124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 114/229] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:30 2022] PUT /identity/v3/projects/d6ff45f877814dada97a096ebf743f75/users/1ac6c9a3c7ae4e34b90fdb3b3111272a/roles/852d973419ee442c80da4214c3df867b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:30.620959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:30.622241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.622723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.623238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.637828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:30.647585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:30.648582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:30.649702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:30.652928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 116/230] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:30.660857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:30.861026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccffa32-7fa6-44ee-abe4-43ecde79b19c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:30.911870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:30.912953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.913475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.913971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.914942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:30.915491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:30.917112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:30.928869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 115/231] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:30.934223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.066392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.129231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4d576-ab21-43a8-8c6b-063bca48fa6c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.195623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.197103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.197584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.198056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] PATH_INFO: `/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.200226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.202945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.206433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 117/232] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:30 2022] GET /identity/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:31.219834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.351671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.468089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56f3ee-ac2d-40bf-bfd2-5054dc5ebbf5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.537611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a57423-0f4f-4af8-9050-d170352e3ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.539685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a57423-0f4f-4af8-9050-d170352e3ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.540473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a57423-0f4f-4af8-9050-d170352e3ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.541234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a57423-0f4f-4af8-9050-d170352e3ad7 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.548514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a57423-0f4f-4af8-9050-d170352e3ad7 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.551466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a57423-0f4f-4af8-9050-d170352e3ad7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.554866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 116/233] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:31 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:31.560992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.674033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b51fe7-8fc8-4894-b63c-0b21a5de30e4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.716479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494d53e-2f96-4749-8016-4cb1c455e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.717405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d53e-2f96-4749-8016-4cb1c455e873 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.717749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d53e-2f96-4749-8016-4cb1c455e873 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.718026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d53e-2f96-4749-8016-4cb1c455e873 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75/users/1ac6c9a3c7ae4e34b90fdb3b3111272a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.724740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494d53e-2f96-4749-8016-4cb1c455e873 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff45f877814dada97a096ebf743f75, user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.726157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494d53e-2f96-4749-8016-4cb1c455e873 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.750897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.757735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 118/234] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:31 2022] PUT /identity/v3/projects/d6ff45f877814dada97a096ebf743f75/users/1ac6c9a3c7ae4e34b90fdb3b3111272a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:32.640835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947554b-49a8-448f-aa75-5b455d61de30 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.641371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947554b-49a8-448f-aa75-5b455d61de30 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.641813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947554b-49a8-448f-aa75-5b455d61de30 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.645176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 117/235] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:32.653131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a5bd2-6aed-4f3a-958e-d7285b87e529 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.653649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a5bd2-6aed-4f3a-958e-d7285b87e529 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.654045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a5bd2-6aed-4f3a-958e-d7285b87e529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.682834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-171a5bd2-6aed-4f3a-958e-d7285b87e529 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:32.684856 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-171a5bd2-6aed-4f3a-958e-d7285b87e529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:32.828677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 119/236] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:32.871570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947554b-49a8-448f-aa75-5b455d61de30 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.102898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5f3b-a784-4f30-b7b1-b471802ba8e0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.103296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5f3b-a784-4f30-b7b1-b471802ba8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.103513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5f3b-a784-4f30-b7b1-b471802ba8e0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.105240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 120/237] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:33.110883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.111205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.111510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.124328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947554b-49a8-448f-aa75-5b455d61de30 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.146447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:33.147683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:33.161359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:33.183352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.184183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.184463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.184734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.199819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:33.211322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:33.212298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:33.213151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:33.218460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 118/238] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:33.219669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.352986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2d53-4b01-4def-a325-5f680849a8a4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.365703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 121/239] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:33.390242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.401336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.402020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.402303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.403247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.404033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:33.404345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:33.405750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:33.416519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 119/240] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:33.422141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.475022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:33.485567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:33.522393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f4dda-1674-45fa-851c-d3340782a9d0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.560475 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caac740-13d8-4252-a773-f44444ba578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.561280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caac740-13d8-4252-a773-f44444ba578e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.561537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24433247-1d3a-4f22-bbb4-f8fa28c1c328 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.561642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caac740-13d8-4252-a773-f44444ba578e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.561832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caac740-13d8-4252-a773-f44444ba578e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.563023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caac740-13d8-4252-a773-f44444ba578e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:33.564530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caac740-13d8-4252-a773-f44444ba578e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:33.566700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 122/241] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:33.572468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce5b23-d0f0-4a1c-b651-89afea58f4f8 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.572760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce5b23-d0f0-4a1c-b651-89afea58f4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.573044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce5b23-d0f0-4a1c-b651-89afea58f4f8 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.574382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 123/242] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:33.575586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e20b04-0246-403c-af18-22d27d895bde None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.575878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e20b04-0246-403c-af18-22d27d895bde None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.576160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e20b04-0246-403c-af18-22d27d895bde None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.578678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 124/243] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:33.578678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e20b04-0246-403c-af18-22d27d895bde None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.602879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a048a-d04e-41a6-982e-38ac37428855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.603863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a048a-d04e-41a6-982e-38ac37428855 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.604077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a048a-d04e-41a6-982e-38ac37428855 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.604356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a048a-d04e-41a6-982e-38ac37428855 None admin] PATH_INFO: `/v3/users/a68f7a66f0744cc9b9c551ff626f4549` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.619029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a048a-d04e-41a6-982e-38ac37428855 None admin] RBAC: Authorizing `identity:get_user(user_id=a68f7a66f0744cc9b9c551ff626f4549)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:33.620543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a048a-d04e-41a6-982e-38ac37428855 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:33.622290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 120/244] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:33 2022] GET /identity/v3/users/a68f7a66f0744cc9b9c551ff626f4549 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:33.623530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767ef7-88bc-4eb0-9572-665e4032c62a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.623816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767ef7-88bc-4eb0-9572-665e4032c62a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.624096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767ef7-88bc-4eb0-9572-665e4032c62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.642730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ec767ef7-88bc-4eb0-9572-665e4032c62a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:33.645717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ec767ef7-88bc-4eb0-9572-665e4032c62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:33.699093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e20b04-0246-403c-af18-22d27d895bde None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.724660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 121/245] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:33.725972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.726265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.726546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.740762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:33.744422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:33.745880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:33.751561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.752251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.752537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.752818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.753939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:33.755930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:33.773292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:33.780089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 125/246] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:33 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:33.788176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.845983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 122/247] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:33.848619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.921588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2808760-1de9-444c-9858-5a3065f7852f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.968006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.968678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.968968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.969251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.003954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae6dc-9e8b-4a92-9ce3-d121bda1d784 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.033808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:34.049547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.051382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.051885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.052426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.059687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:34.061253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:34.067031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:34.075295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:34.076223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:34.077036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:34.079487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 123/248] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:34.080465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.112288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:34.202659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 126/249] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:34.203784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.230966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca22fd-7708-4b8e-a0c3-32f782607314 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.278571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.279937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.280373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.281076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.297816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:34.308410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:34.309395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:34.310173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 124/250] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:34.311447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.313853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:34.341684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56a856-b1da-44b3-9da8-ff4c5f3bd7e4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.400793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.401492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.401777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.402056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.402828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:34.405983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:34.407446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:34.419210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:34.424838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 127/251] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:34.426152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.463973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5062b-cfc7-451a-8eb1-e85f387ad610 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.530483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.531379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.531665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.531944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.533737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:34.535767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:34.538821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 125/252] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:34.539641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.627945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a513c52-882b-44d2-9bcf-aa38273e1e0a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.686160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.688395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.690499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.691829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.692820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:34.693337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:34.694980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:34.709067 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 128/253] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:34.711199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.729305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:34.747234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97ceb-5ef0-44b4-9074-d454e53fc5db None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.796949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.797826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.798108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.798384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] PATH_INFO: `/v3/users/d1e7773dac5a488a81f5cb4297d8f2ef` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.799746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] RBAC: Authorizing `identity:get_user(user_id=d1e7773dac5a488a81f5cb4297d8f2ef)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:34.801458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:34.803440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 126/254] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/users/d1e7773dac5a488a81f5cb4297d8f2ef => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:34.804245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:34.827408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:34.871072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bdc6e-a229-4e55-b275-ed99cf75840b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.919967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:34.921717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.922519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.923018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.924111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] RBAC: Adding query filter params (role.id=f2d22e4432674f50ae6d73b8a5af344b, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=a68f7a66f0744cc9b9c551ff626f4549) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:34.924221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc367c58-bd5c-470c-8c76-32525a7cc32b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:34.924755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:34.927754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2107bd-591a-4d9a-ba25-e3352f9c9109 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:34.936669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 129/255] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/role_assignments?user.id=a68f7a66f0744cc9b9c551ff626f4549&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=f2d22e4432674f50ae6d73b8a5af344b => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:34.938949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de91e-e376-4d9d-89e5-c3562c191c45 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:34.939527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de91e-e376-4d9d-89e5-c3562c191c45 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:34.939964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de91e-e376-4d9d-89e5-c3562c191c45 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:34.947596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 130/256] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:34.948452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7de91e-e376-4d9d-89e5-c3562c191c45 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:35.023972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4a621-1478-4382-b513-a167c93ed474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:35.025063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4a621-1478-4382-b513-a167c93ed474 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.025357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4a621-1478-4382-b513-a167c93ed474 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.025639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4a621-1478-4382-b513-a167c93ed474 None admin] PATH_INFO: `/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.032058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4a621-1478-4382-b513-a167c93ed474 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:35.033549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4a621-1478-4382-b513-a167c93ed474 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:35.035961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 127/257] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:35.037251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.037543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.037825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.063481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:35.094002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:35.099292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:35.099503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7de91e-e376-4d9d-89e5-c3562c191c45 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:35.158435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:35.159439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.160174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.160447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.161658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:35.163713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:35.166058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 131/258] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:34 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:35.167115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:35.224654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 128/259] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:35.225940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:35.273303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:35.316871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75722102-29df-429b-9518-9798d8695ab0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:35.401843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:35.402518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.402805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.403102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.412008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329121d-f81f-4528-8db6-4dbcd6f27217 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:35.418563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:35.466173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:35.467089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.467377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.467648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75/users/d1e7773dac5a488a81f5cb4297d8f2ef/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.471776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff45f877814dada97a096ebf743f75, user_id=d1e7773dac5a488a81f5cb4297d8f2ef, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:35.473769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:35.501978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 129/260] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:35 2022] PUT /identity/v3/projects/d6ff45f877814dada97a096ebf743f75/users/d1e7773dac5a488a81f5cb4297d8f2ef/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:35.503721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:35.543199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:35.549438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:35.620095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:35.672330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e48f-9c98-4c0b-9b03-ced1118aae75 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:35.713465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:35.714332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.714631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.714909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.726587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:35.734848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:35.735859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:35.736665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 130/261] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:35.743841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:35.811013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 132/262] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:35.824420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:35.908339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.017767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49d273-ce79-4ce9-a841-907820daea68 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.106900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35829d2-0702-4995-8657-757791e82e74 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.120974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.130859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.132478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.133251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.134004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.135764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:36.136634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.139621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.153608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.154577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.154932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.155298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.156635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.158252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.158668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 131/263] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.162008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 133/264] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:35 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.171040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.173467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.431991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b93525-3cb0-40a1-8b01-b2075b53b9d0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.448302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe704e-c8cc-4cf3-8cf6-021fa8ac2769 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.468481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.514048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72948f4e-b488-4c0c-9c11-b9965cef7371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.515991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72948f4e-b488-4c0c-9c11-b9965cef7371 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.516714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72948f4e-b488-4c0c-9c11-b9965cef7371 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.517402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72948f4e-b488-4c0c-9c11-b9965cef7371 None admin] PATH_INFO: `/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.520647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72948f4e-b488-4c0c-9c11-b9965cef7371 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac6c9a3c7ae4e34b90fdb3b3111272a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.524807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72948f4e-b488-4c0c-9c11-b9965cef7371 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.528905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 132/265] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/users/1ac6c9a3c7ae4e34b90fdb3b3111272a => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.532527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.533971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.535157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.536671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.538479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] RBAC: Adding query filter params (role.id=852d973419ee442c80da4214c3df867b, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:36.540373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951bfc0-2cbd-4b07-8fe4-6444a39c26e7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.541151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.541862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951bfc0-2cbd-4b07-8fe4-6444a39c26e7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.544875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d8b4-4152-4b05-aff5-96bf4dd285f0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.545349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951bfc0-2cbd-4b07-8fe4-6444a39c26e7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.547137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 133/266] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:36 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:36.550971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951bfc0-2cbd-4b07-8fe4-6444a39c26e7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.572308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 134/267] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/role_assignments?user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=852d973419ee442c80da4214c3df867b => generated 608 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:36.574507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e2ac-e180-4d69-90b2-9cd8a617a0f0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.574970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e2ac-e180-4d69-90b2-9cd8a617a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.575515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e2ac-e180-4d69-90b2-9cd8a617a0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.626216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-27a4e2ac-e180-4d69-90b2-9cd8a617a0f0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:36.633977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-27a4e2ac-e180-4d69-90b2-9cd8a617a0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:36.744607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 135/268] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:36.745340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.747946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb8a6-7ced-443f-bc6c-cc380d7d029c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.748669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb8a6-7ced-443f-bc6c-cc380d7d029c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.749164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb8a6-7ced-443f-bc6c-cc380d7d029c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.751016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 136/269] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:36.762026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.762679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.763192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.794161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:36.796069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:36.855450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951bfc0-2cbd-4b07-8fe4-6444a39c26e7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.915378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.916387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.916881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.917405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.924263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 137/270] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:36.925713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.957735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.006738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.011244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.047348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc55337-9684-4038-8a07-9b227fd97597 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.095362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.096196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.096472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.096779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.099289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.100417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.103353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 138/271] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:37.104243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.119216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.165341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 134/272] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:37.166383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.230636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.234490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578ce66-205d-4b4b-b01f-ce4ab6ea7307 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.278143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.278869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.279175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.279449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.294629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c11971-c629-4256-aa9c-187fdd79fd5a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.295475 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:37.309761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:37.310738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:37.311577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 139/273] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:37.313921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.337895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.339010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.339421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.339800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.340986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.342504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.383629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 135/274] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:37 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:37.384658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.408879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.439611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b525c23-c989-427a-b361-d48da83b308f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.484731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.485618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.485999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.486276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.488623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.490169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.496152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 140/275] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:37 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:37.498209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.512286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6865c3-de9e-4a70-bc86-7ddc35fd0516 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.550574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.551229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.551646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.551987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.552821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:37.553222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.553531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.556044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.566777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 136/276] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:37.572630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.664258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.685396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bb40e-0fab-44c9-a90c-93303d1cb73f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.687027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f05b24-665a-4403-84fa-8a90909e3670 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.725020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.726014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.726365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.726708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] PATH_INFO: `/v3/users/e00743b6dc6345af87c1297adb912425` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.742101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] RBAC: Authorizing `identity:get_user(user_id=e00743b6dc6345af87c1297adb912425)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.743822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.746004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 137/277] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:37 2022] GET /identity/v3/users/e00743b6dc6345af87c1297adb912425 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:37.750888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.751488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:37.751630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.751880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.752152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.753535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:37.753841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.755557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eef337-bffb-4d52-871c-c9052805fdb3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.762183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 141/278] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:37 2022] GET /identity/v3/role_assignments?user.id=1ac6c9a3c7ae4e34b90fdb3b3111272a&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:37.873046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4e563-e531-4e8c-9098-ec3024d4ccad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.917871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.918562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.918843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.919140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.919997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, user.id=e00743b6dc6345af87c1297adb912425, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:37.920309 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.921917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5799ed0-56b1-404f-b338-d4f3dd3ffe6f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.928596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 138/279] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jul 21 22:18:37 2022] GET /identity/v3/role_assignments?user.id=e00743b6dc6345af87c1297adb912425&scope.system=all&role.id=e59097a97ec144ff8463625775da1081 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:37.981730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418bfb1-b0c0-4da9-b49c-f931a743eef1 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.982251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418bfb1-b0c0-4da9-b49c-f931a743eef1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.982251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418bfb1-b0c0-4da9-b49c-f931a743eef1 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.983759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 142/280] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:37.989857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.990158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.990412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.998774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:38.014001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:38.015540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:38.105846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 139/281] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:38.129933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418bfb1-b0c0-4da9-b49c-f931a743eef1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.238414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:38.250633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418bfb1-b0c0-4da9-b49c-f931a743eef1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:38.286759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad4d04-2527-4b8e-b272-86cfa3c4ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:38.287651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad4d04-2527-4b8e-b272-86cfa3c4ad58 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.287937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad4d04-2527-4b8e-b272-86cfa3c4ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.288213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad4d04-2527-4b8e-b272-86cfa3c4ad58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.300303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad4d04-2527-4b8e-b272-86cfa3c4ad58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5bad4d04-2527-4b8e-b272-86cfa3c4ad58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:38.308379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:38.309293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:38.310809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 143/282] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:38.315793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.378443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41af7c8-ed9b-4d14-80a6-9fa48a8704a9 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.379134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41af7c8-ed9b-4d14-80a6-9fa48a8704a9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.379426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41af7c8-ed9b-4d14-80a6-9fa48a8704a9 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.381891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 144/283] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:38.388537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.388835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.389116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.419880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:38.421168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:38.507492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572af0-225a-4c4e-a435-e0fead2582ae None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:38.540946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:38.554145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 145/284] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:38.555128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:38.556073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.556551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.557016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.557958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:38.558457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:38.560101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:38.577894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.582113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 140/285] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:38.587540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.665669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:38.691601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49fe4c-1cc0-4ff2-b66a-81bac2b39dd5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:38.724562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:38.725365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.725648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.725927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.739439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:38.746048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:38.746946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:38.751720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 146/286] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:38.754287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.786084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2082a-cbdc-47e3-9f51-0f69eb536812 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:38.858775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:38.859681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.859968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.860245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] PATH_INFO: `/v3/users/a68f7a66f0744cc9b9c551ff626f4549` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.861648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] RBAC: Authorizing `identity:get_user(user_id=a68f7a66f0744cc9b9c551ff626f4549)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:38.863654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:38.868478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 141/287] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/users/a68f7a66f0744cc9b9c551ff626f4549 => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:38.874022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.921031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0306b3-9309-48e2-ac49-0ec87bc3df32 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:38.956492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:38.957180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:38.957464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:38.957741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:38.958497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:38.958805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:38.967433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:38.969331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 147/288] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:38.974868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:38.977663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.033447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.087496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9303c2-2b3b-47b3-abc4-6c36a6d71d99 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.125220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.126101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.126383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.126658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] PATH_INFO: `/v3/users/d1e7773dac5a488a81f5cb4297d8f2ef` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.128132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] RBAC: Authorizing `identity:get_user(user_id=d1e7773dac5a488a81f5cb4297d8f2ef)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.129598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.131253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 148/289] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/users/d1e7773dac5a488a81f5cb4297d8f2ef => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:39.139894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:39.149971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46efbf-02d6-46f3-ae18-f099bb0a79c3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.228763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.230131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.230635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.231143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.232590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.234312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.243086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 142/290] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:38 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:39.244505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:39.250492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.282960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badb3a8-3d1e-4888-a25b-8fa24d178502 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.321420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.322086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.322376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.322658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.401934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.403848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.515365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 149/291] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:39.515843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.522274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:39.617049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5dde-2153-4e97-a6aa-bad086babaa4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.645871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090a09-558b-4d47-a3ed-2866a1cc46c1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.710305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.722990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d1c8a-5738-4d28-85de-3a75312db710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.724139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d1c8a-5738-4d28-85de-3a75312db710 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.725937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d1c8a-5738-4d28-85de-3a75312db710 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.726393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d1c8a-5738-4d28-85de-3a75312db710 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75/users/a68f7a66f0744cc9b9c551ff626f4549/roles/f2d22e4432674f50ae6d73b8a5af344b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.730002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d1c8a-5738-4d28-85de-3a75312db710 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff45f877814dada97a096ebf743f75, user_id=a68f7a66f0744cc9b9c551ff626f4549, role_id=f2d22e4432674f50ae6d73b8a5af344b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.732105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d1c8a-5738-4d28-85de-3a75312db710 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.738386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.739506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.740083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.740498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.742169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.744285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.746773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 150/292] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:39 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:39.756809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:39.760569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 143/293] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:39 2022] PUT /identity/v3/projects/d6ff45f877814dada97a096ebf743f75/users/a68f7a66f0744cc9b9c551ff626f4549/roles/f2d22e4432674f50ae6d73b8a5af344b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:39.823537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.881924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61562a1b-979b-44bb-8f79-8298fbbcddf7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.924722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.925801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.926285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.926761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.927880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=d1e7773dac5a488a81f5cb4297d8f2ef) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:39.928440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.930231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268edfe-00e9-4460-ae78-b32082f08789 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.937747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 151/294] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:39 2022] GET /identity/v3/role_assignments?user.id=d1e7773dac5a488a81f5cb4297d8f2ef&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=e59097a97ec144ff8463625775da1081 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:40.272029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22485d7e-6330-4944-b61c-6723b18eb92d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.272732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22485d7e-6330-4944-b61c-6723b18eb92d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.273256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22485d7e-6330-4944-b61c-6723b18eb92d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.275482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 144/295] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:40.280858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d033f4-663a-40d6-9918-0d6f815e56c6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.281344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d033f4-663a-40d6-9918-0d6f815e56c6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.281444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d033f4-663a-40d6-9918-0d6f815e56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.303373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-48d033f4-663a-40d6-9918-0d6f815e56c6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:40.304577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-48d033f4-663a-40d6-9918-0d6f815e56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:40.391212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 152/296] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:40.411649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22485d7e-6330-4944-b61c-6723b18eb92d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:40.539410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22485d7e-6330-4944-b61c-6723b18eb92d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:40.557706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4eaf-4d11-409d-b8c6-45b29cbe3fab None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.558360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4eaf-4d11-409d-b8c6-45b29cbe3fab None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.558843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4eaf-4d11-409d-b8c6-45b29cbe3fab None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.560472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 153/297] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:40.566911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288411-e3cf-48f9-a5aa-34c5a9d80fc9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.567306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288411-e3cf-48f9-a5aa-34c5a9d80fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.567833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288411-e3cf-48f9-a5aa-34c5a9d80fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.589316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ac288411-e3cf-48f9-a5aa-34c5a9d80fc9 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:40.590809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92e1c7-c8d2-42cf-8c6c-df49e16ac51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:40.591205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ac288411-e3cf-48f9-a5aa-34c5a9d80fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:40.591653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92e1c7-c8d2-42cf-8c6c-df49e16ac51e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.591940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92e1c7-c8d2-42cf-8c6c-df49e16ac51e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.592223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92e1c7-c8d2-42cf-8c6c-df49e16ac51e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.605359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92e1c7-c8d2-42cf-8c6c-df49e16ac51e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ef92e1c7-c8d2-42cf-8c6c-df49e16ac51e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:40.614081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:40.615028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:40.615856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 145/298] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:40.617032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32774d55-039d-42e6-abfc-1ac04f3bafda None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.617208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32774d55-039d-42e6-abfc-1ac04f3bafda None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.617726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32774d55-039d-42e6-abfc-1ac04f3bafda None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.618893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 146/299] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:40.622211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32774d55-039d-42e6-abfc-1ac04f3bafda None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:40.629006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:40.680204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 154/300] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:40.681728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.682163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.682648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.696979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:40.711782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:40.713634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:40.761032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32774d55-039d-42e6-abfc-1ac04f3bafda None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:40.811222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 155/301] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:40.811222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:40.817382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:40.818349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.818838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.819373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.820333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:40.820840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:40.822439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:40.836915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 147/302] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:40.836915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:40.925833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:40.937596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032ddc3-90b1-498d-842b-17f778779f3d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:40.978016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24b20-50b9-4f00-9687-bdf2c0b91222 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:40.983312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:40.984122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:40.984409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:40.984684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:40.997710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:41.002407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:41.007987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:41.008909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:41.009779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 156/303] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:41.010431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:41.036223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.037066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.037341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.037598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.038858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.041440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.050319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 148/304] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:41.050319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:41.154187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a24f6-d323-43a5-859d-f300ac6ff9ad None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:41.198120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.199012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.199325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.199604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] PATH_INFO: `/v3/users/e00743b6dc6345af87c1297adb912425` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.200895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] RBAC: Authorizing `identity:get_user(user_id=e00743b6dc6345af87c1297adb912425)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.202407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.204363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 157/305] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:41 2022] GET /identity/v3/users/e00743b6dc6345af87c1297adb912425 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:41.205059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:41.213616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51ac0c-14bc-442f-b0ec-9cc76847141f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:41.237359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:41.263572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.264364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.264716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.265048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.265884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:41.266268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.267810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.279818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 149/306] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:41.280778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:41.342090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:41.361301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e63eb-3dd2-4c75-a105-91802a5511df None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:41.407658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.408253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.408524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.408801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] PATH_INFO: `/v3/groups` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.410339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.411893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.420192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823766-2b54-4b3c-a052-656ecd9f27f4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:41.439107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 158/307] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Jul 21 22:18:41 2022] POST /identity/v3/groups => generated 229 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:41.439107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:41.460742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.461633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.461969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.462250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] PATH_INFO: `/v3/system/users/e00743b6dc6345af87c1297adb912425/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.463964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e00743b6dc6345af87c1297adb912425, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.465413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.478905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 150/308] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Jul 21 22:18:41 2022] PUT /identity/v3/system/users/e00743b6dc6345af87c1297adb912425/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:41.591449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:41.634405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b546-72a3-4335-8506-2afc3e019711 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:41.676844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6235d-a8a4-4c18-bef5-b134e4a27bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.677747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6235d-a8a4-4c18-bef5-b134e4a27bfb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.678029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6235d-a8a4-4c18-bef5-b134e4a27bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.678306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6235d-a8a4-4c18-bef5-b134e4a27bfb None admin] PATH_INFO: `/v3/users/2a1a7c67c9474e85b6fd3848ce96e76d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.695811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6235d-a8a4-4c18-bef5-b134e4a27bfb None admin] RBAC: Authorizing `identity:get_user(user_id=2a1a7c67c9474e85b6fd3848ce96e76d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.697321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6235d-a8a4-4c18-bef5-b134e4a27bfb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.699445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 159/309] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:41 2022] GET /identity/v3/users/2a1a7c67c9474e85b6fd3848ce96e76d => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:41.712655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:41.847585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252d08-5f91-4a79-9088-dc241bfa10ec None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:41.859640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:41.891684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:41.892724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:41.893213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:41.894363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:41.895542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] RBAC: Adding query filter params (role.id=f2d22e4432674f50ae6d73b8a5af344b, user.id=2a1a7c67c9474e85b6fd3848ce96e76d, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:41.896064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:41.897810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e246783-90b7-47e9-a935-a728f377ee44 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:41.904657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 151/310] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jul 21 22:18:41 2022] GET /identity/v3/role_assignments?user.id=2a1a7c67c9474e85b6fd3848ce96e76d&scope.system=all&role.id=f2d22e4432674f50ae6d73b8a5af344b => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:42.240133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8baee-9386-42e1-99b6-079b07cf845c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:42.242078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8baee-9386-42e1-99b6-079b07cf845c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:42.242078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8baee-9386-42e1-99b6-079b07cf845c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:42.242514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 160/311] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:42.248317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:42.248877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:42.249383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:42.267293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:42.268909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:42.344846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 152/312] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:42.370233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8baee-9386-42e1-99b6-079b07cf845c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:42.481210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8baee-9386-42e1-99b6-079b07cf845c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:42.517909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:42.519223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:42.519669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:42.519948 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:42.528085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:42.532334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:42.540612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:42.541575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:42.542389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 161/313] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:42.549327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:42.691469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:42.703050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d66dbf-c374-4727-80c4-55ae56e5d341 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:42.744346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:42.745132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:42.745295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:42.745564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:42.746325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:42.746634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:42.748131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:42.758332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 153/314] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:42.763752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:42.874343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ede0-e60b-44e8-bed2-cc6fb24a3535 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:42.910858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:42.911778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:42.912061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:42.912339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] PATH_INFO: `/v3/users/a68f7a66f0744cc9b9c551ff626f4549` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:42.913582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] RBAC: Authorizing `identity:get_user(user_id=a68f7a66f0744cc9b9c551ff626f4549)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:42.915068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:42.917026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 162/315] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:42 2022] GET /identity/v3/users/a68f7a66f0744cc9b9c551ff626f4549 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:42.922474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:43.033582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585b7ed-f4ee-4b14-b67e-35c9272d8ec5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:43.073719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:43.074376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:43.074660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:43.074936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:43.107513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:43.152330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:43.153738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:43.261951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 154/316] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:43.267260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:43.347276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:43.390121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb03e-0506-43a4-8f4b-b6c81f4eb34c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:43.435646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972562c-fb50-4018-99e3-db0986591434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:43.437045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972562c-fb50-4018-99e3-db0986591434 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:43.437550 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972562c-fb50-4018-99e3-db0986591434 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:43.438055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972562c-fb50-4018-99e3-db0986591434 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:43.441245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972562c-fb50-4018-99e3-db0986591434 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:43.444131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972562c-fb50-4018-99e3-db0986591434 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:43.447267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 163/317] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:43 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:43.457040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:43.555277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:43.599095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2508f-5be0-411f-9d75-709d2913ae9a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:43.636933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:43.637611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:43.637895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:43.638173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:43.639021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] RBAC: Adding query filter params (role.id=f2d22e4432674f50ae6d73b8a5af344b, scope.project.id=d6ff45f877814dada97a096ebf743f75, user.id=a68f7a66f0744cc9b9c551ff626f4549) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:43.639430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:43.641043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325464e3-f72e-4baa-a5fc-3c481fb0b98b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:43.649252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 155/318] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:43 2022] GET /identity/v3/role_assignments?user.id=a68f7a66f0744cc9b9c551ff626f4549&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=f2d22e4432674f50ae6d73b8a5af344b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:44.106776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66aa4-f853-4206-aaa1-84abbb38787d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.107332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66aa4-f853-4206-aaa1-84abbb38787d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.107840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66aa4-f853-4206-aaa1-84abbb38787d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.110488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 164/319] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.116654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72686c4-628a-485a-8582-91b0d55a5cdc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.117451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72686c4-628a-485a-8582-91b0d55a5cdc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.117451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72686c4-628a-485a-8582-91b0d55a5cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.132379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26604f-f6b6-4f59-a9e9-e30113212011 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.132894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26604f-f6b6-4f59-a9e9-e30113212011 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.133408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26604f-f6b6-4f59-a9e9-e30113212011 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.135539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b72686c4-628a-485a-8582-91b0d55a5cdc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:44.136158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 165/320] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.136781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b72686c4-628a-485a-8582-91b0d55a5cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:44.145757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.146272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.146758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.173682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:44.175641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:44.213462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 156/321] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:44.215332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c277-861c-4ee4-a076-458dbd5b86bb None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.215794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c277-861c-4ee4-a076-458dbd5b86bb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.216236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49c277-861c-4ee4-a076-458dbd5b86bb None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.218472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 157/322] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.227324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.227786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.228232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.254367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:44.256291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:44.274412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 166/323] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:44.275758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.329533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.349240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 158/324] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:44.350198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.366911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.429537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d780cf-38a5-4482-b124-b1741d2feec5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.467959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b8555-918f-4919-9f8d-1d4465b9220d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.473562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.474497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.474865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.475267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.489603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:44.498993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:44.501058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:44.503661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 167/325] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:44.505283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.515456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.516235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.516517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.516799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.533188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:44.541173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:44.542965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 159/326] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:44.544847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.621884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.673560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1b65d-b46a-46ef-820f-63184222e0ae None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.696588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3868eb6-b1ce-46dd-b687-1bcca581c083 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.712383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.728401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.729416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.729821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.730260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.731756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:44.733523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:44.735684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 168/327] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:44.736750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.746799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.747532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.747816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.748091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.749466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:44.749793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:44.751257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:44.762559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 160/328] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:44.763428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.872532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3521e-5de1-40b0-ac9e-6d6d850cd1e2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.886456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f9b6c-c816-4943-bcab-b654301a99cf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.896768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.909738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.910775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.910775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.911112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.911699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:44.912005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:44.913431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:44.922171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 169/329] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:44.922915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.931460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.932152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.932434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.932718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] PATH_INFO: `/v3/groups` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.933425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:44.934909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:44.953777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 161/330] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/groups => generated 226 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:44.954838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.002304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.063474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04036ae5-c988-4616-825e-d019e568c314 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.101168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cb700-87de-4683-ba3e-3e35b53d99a1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.104901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.105798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.106080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.106357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] PATH_INFO: `/v3/users/2a1a7c67c9474e85b6fd3848ce96e76d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.107688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] RBAC: Authorizing `identity:get_user(user_id=2a1a7c67c9474e85b6fd3848ce96e76d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.109283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.111303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 170/331] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/users/2a1a7c67c9474e85b6fd3848ce96e76d => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:45.117532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.159715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.160845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.161140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.161638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] PATH_INFO: `/v3/users/e00743b6dc6345af87c1297adb912425` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.165379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] RBAC: Authorizing `identity:get_user(user_id=e00743b6dc6345af87c1297adb912425)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.168100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.171874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 162/332] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/users/e00743b6dc6345af87c1297adb912425 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:45.181911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.231825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.272558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8c80a-4f0b-404c-b33c-eddf19ddb33f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.327394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.341492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8bb3-a160-46c0-a960-782b19f7d170 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.343586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7118-e20d-4717-ad47-20f139efc415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.344654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7118-e20d-4717-ad47-20f139efc415 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.344946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7118-e20d-4717-ad47-20f139efc415 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.345223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7118-e20d-4717-ad47-20f139efc415 None admin] PATH_INFO: `/v3/system/users/2a1a7c67c9474e85b6fd3848ce96e76d/roles/f2d22e4432674f50ae6d73b8a5af344b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.348132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d7118-e20d-4717-ad47-20f139efc415 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2a1a7c67c9474e85b6fd3848ce96e76d, role_id=f2d22e4432674f50ae6d73b8a5af344b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.349600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d7118-e20d-4717-ad47-20f139efc415 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.366443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 171/333] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Jul 21 22:18:45 2022] PUT /identity/v3/system/users/2a1a7c67c9474e85b6fd3848ce96e76d/roles/f2d22e4432674f50ae6d73b8a5af344b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:45.391229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.391992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.392279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.392553 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.394372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] RBAC: Adding query filter params (role.id=e59097a97ec144ff8463625775da1081, user.id=e00743b6dc6345af87c1297adb912425, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:45.394789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.396518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed742d-5218-4c6f-9964-8cb8304dae36 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.417208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 163/334] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/role_assignments?user.id=e00743b6dc6345af87c1297adb912425&scope.system=all&role.id=e59097a97ec144ff8463625775da1081 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:47.686166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3514d2-fea0-48ce-be7f-a3ca04d1e781 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:47.686690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3514d2-fea0-48ce-be7f-a3ca04d1e781 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:47.686803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3514d2-fea0-48ce-be7f-a3ca04d1e781 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:47.688161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 172/335] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:47.697576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bd79c-d15e-4d98-a9fd-87edbfd9a9df None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:47.697576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bd79c-d15e-4d98-a9fd-87edbfd9a9df None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:47.697576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bd79c-d15e-4d98-a9fd-87edbfd9a9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:47.727405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-397bd79c-d15e-4d98-a9fd-87edbfd9a9df None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:47.728697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-397bd79c-d15e-4d98-a9fd-87edbfd9a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:47.824488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 164/336] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:47.874089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3514d2-fea0-48ce-be7f-a3ca04d1e781 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.153619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71be1d-6b5a-48de-83f3-f524b7d9619e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.154337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71be1d-6b5a-48de-83f3-f524b7d9619e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.154860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71be1d-6b5a-48de-83f3-f524b7d9619e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.160830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3514d2-fea0-48ce-be7f-a3ca04d1e781 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.161373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 165/337] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:48.162921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0e0f-6e6a-41b4-a2af-d439cd1c0066 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.167778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0e0f-6e6a-41b4-a2af-d439cd1c0066 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.168330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0e0f-6e6a-41b4-a2af-d439cd1c0066 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.170321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 166/338] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:48.172606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.173130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.173611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.228487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:48.231951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:48.237671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daec48b-0592-4e78-8635-3ce0ba84bd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.239247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daec48b-0592-4e78-8635-3ce0ba84bd57 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.240325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daec48b-0592-4e78-8635-3ce0ba84bd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.240778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daec48b-0592-4e78-8635-3ce0ba84bd57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.259205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daec48b-0592-4e78-8635-3ce0ba84bd57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6daec48b-0592-4e78-8635-3ce0ba84bd57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:48.269296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:48.270346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:48.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:48.274839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 173/339] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:48.277313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.280530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.280530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.302963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:48.304250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:48.306422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.363703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.404161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 174/340] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:48.404959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.473812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 167/341] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:48.474751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.621982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35517bdb-b556-4483-a552-f4595965d1d1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.622209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07d521-2d19-4b3b-9a2a-6e661c66eb3a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.671249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4774c6f-97e0-47db-996c-d8146479b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.672429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4774c6f-97e0-47db-996c-d8146479b81d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.672938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4774c6f-97e0-47db-996c-d8146479b81d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.673407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4774c6f-97e0-47db-996c-d8146479b81d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.687608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4774c6f-97e0-47db-996c-d8146479b81d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c4774c6f-97e0-47db-996c-d8146479b81d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:48.696678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:48.697622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:48.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:48.701688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 168/342] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:48.703388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3d644-85d5-41ab-9b89-e4ac61c4567f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.703896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3d644-85d5-41ab-9b89-e4ac61c4567f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.704390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3d644-85d5-41ab-9b89-e4ac61c4567f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.706828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 169/343] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:48.707946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3d644-85d5-41ab-9b89-e4ac61c4567f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.718988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.720164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.720665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.721784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.723224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:48.724661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:48.728093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:48.735945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.751685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 175/344] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:48.754250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.825208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.909660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3d644-85d5-41ab-9b89-e4ac61c4567f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.934164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be4ef-0b91-4709-a5f7-8617a55ff948 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.955727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfc796-af54-4cde-825b-16732dc8bdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.957044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfc796-af54-4cde-825b-16732dc8bdab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.958161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfc796-af54-4cde-825b-16732dc8bdab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.958663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfc796-af54-4cde-825b-16732dc8bdab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.972985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfc796-af54-4cde-825b-16732dc8bdab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-87cfc796-af54-4cde-825b-16732dc8bdab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:48.982706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:48.983715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:48.984597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:48.987475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 170/345] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:48.989285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.989733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.990169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.016870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:49.018328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.018874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.019570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.021129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:49.021380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:49.021920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:49.022886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:49.025178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:49.043040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 176/346] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:49.045276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:49.118657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 171/347] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:49.119561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:49.124116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:49.248797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:49.254417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e3778-850a-449e-ae6c-335eaffc763a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:49.283996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edda0b4-0a96-404b-b15f-4492216d1f61 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:49.298404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:49.299123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.300079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.300414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.358044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:49.361173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.363363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.364258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.366135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:49.367003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:49.370130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:49.390050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 172/348] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:49.392097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:49.398234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:49.400749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:49.516947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:49.522639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:49.530648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 177/349] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:49.531888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:49.569305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24aedd-0197-4651-a51a-1000adc83162 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:49.622949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301d18d-ae01-4d02-965c-d4e3f3009386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:49.623657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301d18d-ae01-4d02-965c-d4e3f3009386 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.623946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301d18d-ae01-4d02-965c-d4e3f3009386 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.624221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301d18d-ae01-4d02-965c-d4e3f3009386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.698974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac3799-03ba-4490-a5a2-74b1ebd5bfd7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:49.740170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301d18d-ae01-4d02-965c-d4e3f3009386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:49.742480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301d18d-ae01-4d02-965c-d4e3f3009386 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:49.762024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:49.764336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.765078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.765783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.791884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:49.802951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:49.804579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:49.805999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 178/350] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:49.807781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:49.848425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:49.915531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:49.918031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 173/351] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:49.920109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f13a5-7719-4a84-b321-d99f3eeb473c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.920401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f13a5-7719-4a84-b321-d99f3eeb473c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.920675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f13a5-7719-4a84-b321-d99f3eeb473c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.922105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 174/352] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:49.922832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f13a5-7719-4a84-b321-d99f3eeb473c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:49.993281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de559e2-b35b-4ebe-8747-061e2e24d618 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.056340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.057764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.058337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.058850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] PATH_INFO: `/v3/users/2a1a7c67c9474e85b6fd3848ce96e76d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.061033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] RBAC: Authorizing `identity:get_user(user_id=2a1a7c67c9474e85b6fd3848ce96e76d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.062937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.066842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 179/353] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/users/2a1a7c67c9474e85b6fd3848ce96e76d => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.068338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.095983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f13a5-7719-4a84-b321-d99f3eeb473c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.157523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07468dd-d941-4192-b346-add16b49693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.160602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07468dd-d941-4192-b346-add16b49693e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.161369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07468dd-d941-4192-b346-add16b49693e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.162123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07468dd-d941-4192-b346-add16b49693e None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.164479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07468dd-d941-4192-b346-add16b49693e None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.167465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07468dd-d941-4192-b346-add16b49693e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.178187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 175/354] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.180330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07468dd-d941-4192-b346-add16b49693e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.220993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bc423-7710-4afa-939a-225cad00bd2b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.243107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.259996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.282244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.283914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.284861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.284861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.286170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:50.286668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.290504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588f0cd-2312-4f1f-a646-cd33eb51e71e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.311382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 180/355] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.318870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.318870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.318870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.347343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:50.349264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:50.368113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07468dd-d941-4192-b346-add16b49693e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.439422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.440963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.441444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.441915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.444230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.446975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.450473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 176/356] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.452024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.493849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 181/357] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:50.495638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.646792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d61-1d97-40ff-b008-92b4b84eb58f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.651106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.677076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67eec8b-8d2a-40fc-8ad2-830c54b1b24d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.714260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.726045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.727224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.727729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.728224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.729819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] RBAC: Adding query filter params (role.id=f2d22e4432674f50ae6d73b8a5af344b, user.id=2a1a7c67c9474e85b6fd3848ce96e76d, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:50.730385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.733235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.737919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.740182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.740956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.741707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.753940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 177/358] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/role_assignments?user.id=2a1a7c67c9474e85b6fd3848ce96e76d&scope.system=all&role.id=f2d22e4432674f50ae6d73b8a5af344b => generated 510 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.755359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.757108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.760542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.764299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 182/359] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.771262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.964091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f0ec0-285a-4d0c-9774-20b00b16bdb7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.977576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ba3de-9e53-446d-823d-cb27e6a7987e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.021351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.022837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.023375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.024129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.026283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.029312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.032383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 183/360] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:51.039023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.039298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.039298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.039298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.039298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.073868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.123260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.141763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.145925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.214875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe5df5-e67f-4bd8-abaa-e5fd4e3c68c6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.273495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa6de-096b-4d1b-9e51-7ff68d04d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.274776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa6de-096b-4d1b-9e51-7ff68d04d89f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.275236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa6de-096b-4d1b-9e51-7ff68d04d89f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.275654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa6de-096b-4d1b-9e51-7ff68d04d89f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.294679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fa6de-096b-4d1b-9e51-7ff68d04d89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-df3fa6de-096b-4d1b-9e51-7ff68d04d89f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:51.305734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:51.308880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 184/361] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:51.310425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3c7f-e920-4545-a991-c082eff4f55b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.312396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3c7f-e920-4545-a991-c082eff4f55b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.312396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3c7f-e920-4545-a991-c082eff4f55b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.313368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 185/362] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:51.314380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3c7f-e920-4545-a991-c082eff4f55b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.352395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 178/363] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:51.354081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.467287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.517445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3c7f-e920-4545-a991-c082eff4f55b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.534688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb59ec-eee8-4edc-a665-13208f950b56 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.542193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.578028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.578710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.579085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.579363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.580883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=e59097a97ec144ff8463625775da1081, scope.project.id=3be9f13173874417a10d07a7e522883a) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:51.581632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.581818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.582718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.582816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d8b5c-f383-4e1d-b17f-1b1d86b375e5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.583302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.583899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.586125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=e59097a97ec144ff8463625775da1081, scope.project.id=d6ff45f877814dada97a096ebf743f75) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:51.586270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.589052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.595484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 186/364] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=e59097a97ec144ff8463625775da1081 => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:51.597389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.597721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.598005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.601416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 179/365] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=e59097a97ec144ff8463625775da1081 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:51.603179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.620960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:51.622677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:51.733655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 187/366] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:51.734474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.857105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f38d50-c38a-430b-b4c3-5c56891dffac None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.902432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7cf4c-5b2c-4832-bf4d-1d128c8d0c26 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.917162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.921736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.952930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.954161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.954739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.957088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.958385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:51.958945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.961536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.980185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 180/367] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:51.981707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.993135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.994523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.995008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.995556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.997884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.000693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.004307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 188/368] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:52.005835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.217195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434d13f-4022-4dba-a753-128951fef03d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.253783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b2184-c7bc-4934-9326-7bbb06ca4cab None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.296338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.297696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.298204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.298549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.323719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:52.337177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:52.339403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:52.341379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:52.342792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.342792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.342792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.343223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 181/369] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:52.343346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.343470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.434007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.446620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.504040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.506925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.541731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edf3e1-8719-485f-8aaa-11170797511f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.626027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.629020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.629535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.630399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.633180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.636610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.640919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 182/370] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:52.642472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.758332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 189/371] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 753 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:52.759701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.856727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d765be5-de6c-46b9-bee3-e612e860d824 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.885011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.927857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.934955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.936207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.936714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.938110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.939860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:52.940393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.943976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.954996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea0407-178f-4b9e-b689-a68438935314 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.962183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 183/372] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:52.971584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.000344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.001250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.001632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.002077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.003094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=852d973419ee442c80da4214c3df867b, scope.project.id=3be9f13173874417a10d07a7e522883a) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:53.003535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.005249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.012606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 190/373] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=852d973419ee442c80da4214c3df867b => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:53.013615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.131586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b43cb5-e870-4d56-93f6-d232d4abbfd3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.144872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0c8b3-182c-4615-a6bc-4fd94678420d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.181110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.181664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.181951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.182228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.187025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.188284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.188771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.189270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.191012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.192862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.194879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 184/374] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:53.196012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.206320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.267437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.269240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.366446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.389349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 191/375] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:53.398262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.474739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.499712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570685-10a7-4d47-86c9-43ec489434cb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.565502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.566930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.567458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.567977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.570133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.572767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.579085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 185/376] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:53.589580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.619304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b252604-68bb-4f7f-9d51-2d3a7f75ceac None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.688698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.690123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.690405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.690680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] PATH_INFO: `/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.693731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=1a2e5fa7e325464b9d8cfafd39af8ec6)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.698841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.704253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 192/377] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6 => generated 344 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:53.707978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.804102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92ad62-e316-4c96-bb3a-62ae7b8bcd9f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.833249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.859551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.860587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.861077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.861587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.862673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=852d973419ee442c80da4214c3df867b, scope.project.id=d6ff45f877814dada97a096ebf743f75) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:53.863260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.864937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.871940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 186/378] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=852d973419ee442c80da4214c3df867b => generated 260 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:53.915622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33f427-be76-41a3-90d4-8e1ce274f1e7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.929166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.998782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f8e45-8330-4b30-9d49-d3a035fa75f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.000365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f8e45-8330-4b30-9d49-d3a035fa75f9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.000826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f8e45-8330-4b30-9d49-d3a035fa75f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.001327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f8e45-8330-4b30-9d49-d3a035fa75f9 None admin] PATH_INFO: `/v3/groups/b0ea0594926440aa9628cb1924af59f8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.020567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f8e45-8330-4b30-9d49-d3a035fa75f9 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ea0594926440aa9628cb1924af59f8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.023199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f8e45-8330-4b30-9d49-d3a035fa75f9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.026848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 193/379] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/groups/b0ea0594926440aa9628cb1924af59f8 => generated 229 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.036051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.296359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff8c9-8f2a-471e-8c66-44f677fa12b7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.361880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.363041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.363589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.364095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.365781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] RBAC: Adding query filter params (group.id=b0ea0594926440aa9628cb1924af59f8, role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=1a2e5fa7e325464b9d8cfafd39af8ec6) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:54.366356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.369206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f90e5-cf23-4eaa-820a-54a2a93a8238 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.381214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 187/380] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/role_assignments?group.id=b0ea0594926440aa9628cb1924af59f8&scope.project.id=1a2e5fa7e325464b9d8cfafd39af8ec6&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 260 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.449799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333668-e81f-4735-ba0f-e518b8545535 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.450139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333668-e81f-4735-ba0f-e518b8545535 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.450465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333668-e81f-4735-ba0f-e518b8545535 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.453213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 194/381] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:54.460843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7811b-48ab-4a10-94e5-47c6c98a7ec2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.461353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7811b-48ab-4a10-94e5-47c6c98a7ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.461858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7811b-48ab-4a10-94e5-47c6c98a7ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.492560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c9f7811b-48ab-4a10-94e5-47c6c98a7ec2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:54.493877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c9f7811b-48ab-4a10-94e5-47c6c98a7ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:54.567124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:54.618216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 188/382] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:54.646019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333668-e81f-4735-ba0f-e518b8545535 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.797139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d353f-98cd-4d14-b852-aa0cdff7d8aa None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.797711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d353f-98cd-4d14-b852-aa0cdff7d8aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.798083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d353f-98cd-4d14-b852-aa0cdff7d8aa None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.802002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 189/383] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:54.807631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.808096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.808588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.845428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:54.851995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:54.862655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333668-e81f-4735-ba0f-e518b8545535 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.957874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.958966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.959127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.959506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.980917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:54.993112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:54.994932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:54.996620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 195/384] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:55.000981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 190/385] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:55.008818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.055502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.072733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.139369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1a22f-1242-441f-bd31-596a02d3e2ef None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.180748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.181017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.181353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.181637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.185537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:55.185537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.185537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.195181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 196/386] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.206994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.318562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.326058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c86c-2823-4b5b-8749-2a8b3dd577c6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.375447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.392468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6683a-1920-4282-80c2-7e450e8dad1c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.408210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.409430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.409849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.410135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.454119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:55.469589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.470456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.475613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.476008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.477352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.478901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:55.480153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:55.481766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:55.483367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:55.484569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 197/387] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.497751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 191/388] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:55.504223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.508702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.640929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28094ca-b0a6-43a6-b9c0-20696fe48363 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.681306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.682193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.682474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.682750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.684047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.685524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.687653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 198/389] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.708815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.789104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c300f-17fa-42cd-bf36-a9e580740239 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.815960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.878669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.879394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.879683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.879970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.880742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:55.881057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.882505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.911283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 192/390] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.919845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.982881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048f577-4472-42cb-8c9f-c035940f9d30 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.985257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.038016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13924b8a-e3e7-4bb6-a85e-8e25cc3163da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.038920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924b8a-e3e7-4bb6-a85e-8e25cc3163da None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.039230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924b8a-e3e7-4bb6-a85e-8e25cc3163da None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.039504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924b8a-e3e7-4bb6-a85e-8e25cc3163da None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75/groups/515033a21229473a9ebee545c0cde34e/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.043676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924b8a-e3e7-4bb6-a85e-8e25cc3163da None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff45f877814dada97a096ebf743f75, group_id=515033a21229473a9ebee545c0cde34e, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.045147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924b8a-e3e7-4bb6-a85e-8e25cc3163da None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.074434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 199/391] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jul 21 22:18:55 2022] PUT /identity/v3/projects/d6ff45f877814dada97a096ebf743f75/groups/515033a21229473a9ebee545c0cde34e/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:56.075929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa51b-1bd9-4382-b8e9-780b86a26835 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.076278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa51b-1bd9-4382-b8e9-780b86a26835 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.077033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa51b-1bd9-4382-b8e9-780b86a26835 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.078968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 200/392] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:56.102146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.102641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.102892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.140495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dedaa-7fb6-4a82-b9dd-2d997b865d82 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.144382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:56.145696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:56.186582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.187496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.187765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.188049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.189508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.191610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.193800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 193/393] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:56.202760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.339848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 201/394] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 22:18:56.385727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.419940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.423498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.433480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec05753-1eed-4233-94aa-ac1aa0098041 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.484988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.485909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.486818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.487128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.491941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.493564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.495912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 194/395] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:56 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:56.521986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.625447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86f9d3-ae10-4ee1-a4eb-fcc56a166fd1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.711757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.712587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.712869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.713145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.743836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:56.747823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4949bb4-0aec-4534-993f-941e175408cf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:56.753780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:56.754753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:56.755595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:56.767140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 202/396] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:56.769169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.787881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919950ca-67e0-4c9d-8e42-452c3c5c906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.788790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919950ca-67e0-4c9d-8e42-452c3c5c906f None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.789074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919950ca-67e0-4c9d-8e42-452c3c5c906f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.789349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919950ca-67e0-4c9d-8e42-452c3c5c906f None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a/groups/515033a21229473a9ebee545c0cde34e/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.791593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919950ca-67e0-4c9d-8e42-452c3c5c906f None admin] RBAC: Authorizing `identity:create_grant(project_id=3be9f13173874417a10d07a7e522883a, group_id=515033a21229473a9ebee545c0cde34e, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.793102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919950ca-67e0-4c9d-8e42-452c3c5c906f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.817932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 195/397] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jul 21 22:18:56 2022] PUT /identity/v3/projects/3be9f13173874417a10d07a7e522883a/groups/515033a21229473a9ebee545c0cde34e/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:56.820951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e270-859f-40b2-ac7c-18fa7cf217e2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.823453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e270-859f-40b2-ac7c-18fa7cf217e2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.824366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e270-859f-40b2-ac7c-18fa7cf217e2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.826715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 196/398] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:56.835020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.835687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.836175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.869907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:56.872541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:56.905707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.932116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.002993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b9251-c75f-4d52-a4a4-ba56ba6c6268 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.046170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 197/399] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:57.085094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.086280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.086786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.087325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.088832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:57.089372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.092168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.102706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.113102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 203/400] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:57.123549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.261750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8438f-b8f5-4f8c-8682-0f026caeb49e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.300873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0bf31-95d9-4420-84c1-3e2b349f9ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.301698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0bf31-95d9-4420-84c1-3e2b349f9ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.301980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0bf31-95d9-4420-84c1-3e2b349f9ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.302261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0bf31-95d9-4420-84c1-3e2b349f9ff8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.315508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0bf31-95d9-4420-84c1-3e2b349f9ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-40f0bf31-95d9-4420-84c1-3e2b349f9ff8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:57.324062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:57.325853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 198/401] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:57.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3555e9-a79d-4eab-aff3-6cc0545181c6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.334865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3555e9-a79d-4eab-aff3-6cc0545181c6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.334865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3555e9-a79d-4eab-aff3-6cc0545181c6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.334865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 199/402] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:57.334865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d654788-5700-4454-97b1-185407d92578 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.339001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d654788-5700-4454-97b1-185407d92578 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.339001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d654788-5700-4454-97b1-185407d92578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.351241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5ca86-551f-403a-bcee-7b0f8ed8ba49 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.359692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7d654788-5700-4454-97b1-185407d92578 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:57.360921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7d654788-5700-4454-97b1-185407d92578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:57.363792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.391996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.393706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.394273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.394696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.396351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.398010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.400701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 204/403] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:57.401941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.415343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.439187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 200/404] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:57.440894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d654788-5700-4454-97b1-185407d92578 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.556313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cba8fa-fe1b-427f-b0f1-d3d680301e6e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.583172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d654788-5700-4454-97b1-185407d92578 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.603604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.604484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.604773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.605053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.606205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:57.606521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.607990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.621953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 205/405] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:57.622829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.626315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.628375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.628931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.629469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.631101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.633042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.635131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 201/406] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:57.636343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.647580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.723193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.770539 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd152-08ad-4ec9-b001-a9cb38cf3bcc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.771832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025bfa-b37b-4389-a2a1-972c75b3d54a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.817910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.818635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.818925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.819753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.820610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.820890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.821163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.821895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.823037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.824575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.831869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 202/407] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:57.831869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.846745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:57.857186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:57.858150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:57.858954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:57.863127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 206/408] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:57.864594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.927104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.956994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a79f7-c98d-48c8-9e10-a09ad34f3517 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.006200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.007149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.007441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.007720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a/groups/515033a21229473a9ebee545c0cde34e/roles/852d973419ee442c80da4214c3df867b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.008951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d43342-71ad-44da-9fdc-cd119c2cf3c7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.010263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be9f13173874417a10d07a7e522883a, group_id=515033a21229473a9ebee545c0cde34e, role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.012180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.034941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 203/409] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jul 21 22:18:57 2022] PUT /identity/v3/projects/3be9f13173874417a10d07a7e522883a/groups/515033a21229473a9ebee545c0cde34e/roles/852d973419ee442c80da4214c3df867b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:58.036068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.055464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:58.071544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.072503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.072894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.073276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.080736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.082727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.087176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 207/410] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:58.102579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.180797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0631d2-0ad4-44a5-84ad-5d3809e84442 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.244119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8ae6-1429-418d-bae2-9561664a5348 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.249561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.250252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.250536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.250812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.252447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:58.252813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.254360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.256073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:58.266054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 204/411] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:58.285506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.306751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.308128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.308692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.309225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75/groups/515033a21229473a9ebee545c0cde34e/roles/852d973419ee442c80da4214c3df867b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.317479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff45f877814dada97a096ebf743f75, group_id=515033a21229473a9ebee545c0cde34e, role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.319470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.366212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 208/412] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jul 21 22:18:58 2022] PUT /identity/v3/projects/d6ff45f877814dada97a096ebf743f75/groups/515033a21229473a9ebee545c0cde34e/roles/852d973419ee442c80da4214c3df867b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:58.464383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455192-effc-499a-8e6d-14c149a06f06 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.543319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.545053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.545560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.546053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] PATH_INFO: `/v3/groups/b0ea0594926440aa9628cb1924af59f8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.548330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] RBAC: Authorizing `identity:get_group(group_id=b0ea0594926440aa9628cb1924af59f8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.553098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.556933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 205/413] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:18:58 2022] GET /identity/v3/groups/b0ea0594926440aa9628cb1924af59f8 => generated 229 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:58.562634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.683433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:58.771259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470efec8-175c-4b64-894f-05db6921ee39 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.842715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4c6eb-fa52-4ea6-8541-0eea94f8a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.844231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4c6eb-fa52-4ea6-8541-0eea94f8a85e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.844735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4c6eb-fa52-4ea6-8541-0eea94f8a85e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.845238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4c6eb-fa52-4ea6-8541-0eea94f8a85e None admin] PATH_INFO: `/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.849055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4c6eb-fa52-4ea6-8541-0eea94f8a85e None admin] RBAC: Authorizing `identity:get_project(project_id=1a2e5fa7e325464b9d8cfafd39af8ec6)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.851864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4c6eb-fa52-4ea6-8541-0eea94f8a85e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.855485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 209/414] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:58 2022] GET /identity/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6 => generated 344 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:58.865059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.994533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:59.040442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777518-0931-456d-ad87-366695fcda9d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:59.081312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36c40f-95d0-47bb-973e-ec736cc61553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:59.082220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36c40f-95d0-47bb-973e-ec736cc61553 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.082506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36c40f-95d0-47bb-973e-ec736cc61553 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.082779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36c40f-95d0-47bb-973e-ec736cc61553 None admin] PATH_INFO: `/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6/groups/b0ea0594926440aa9628cb1924af59f8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.084887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36c40f-95d0-47bb-973e-ec736cc61553 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a2e5fa7e325464b9d8cfafd39af8ec6, group_id=b0ea0594926440aa9628cb1924af59f8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:59.086367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36c40f-95d0-47bb-973e-ec736cc61553 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:59.111642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 206/415] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jul 21 22:18:58 2022] PUT /identity/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6/groups/b0ea0594926440aa9628cb1924af59f8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:18:59.484014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cac7b-1df5-40b7-9ac9-d1e3c8b8e0ff None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.484691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cac7b-1df5-40b7-9ac9-d1e3c8b8e0ff None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.484988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cac7b-1df5-40b7-9ac9-d1e3c8b8e0ff None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.489606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 210/416] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:18:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:59.497002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.497632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.497956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.536660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:59.538686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:59.631657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 207/417] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:59.665163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cac7b-1df5-40b7-9ac9-d1e3c8b8e0ff None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:59.841700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cac7b-1df5-40b7-9ac9-d1e3c8b8e0ff None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:59.881754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:59.903875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:59.905264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.905757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.906259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.933217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:59.948824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:59.949807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:59.950627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:18:59.951816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 211/418] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:59.959073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:00.065997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:00.135788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433fdb-cd7a-4eee-acde-2b4e1ccc5b23 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:00.174901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:00.175621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.175910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.176190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.177125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:00.177444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:00.178873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701bf1d-0e98-401d-bae6-3ae23c82b197 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:00.188468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 208/419] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:00.195701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:00.389860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab216d-6dfe-49f0-a6fe-bf9c13cd7d39 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:00.464627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:00.465652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.466114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.466539 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.570599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:00.591249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:00.592757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:00.694358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99695c-48fb-4b70-9c6f-ef2d739462fd None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.695391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99695c-48fb-4b70-9c6f-ef2d739462fd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.695875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99695c-48fb-4b70-9c6f-ef2d739462fd None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.697559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 209/420] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:00.709956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.710584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.711107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.731259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:00.733057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:00.837403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 212/421] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:00.841393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:00.902806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 210/422] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:00.930517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.019350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.027045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.044441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb9fe7-fe48-41b2-8ee4-0b0d10bafbd0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.096250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.099628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.100195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.101715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.103765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.106335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.114914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 213/423] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:00 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:01.117494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.149155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66805f0-128c-48e4-977b-08125d65a925 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.235505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.237162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.237686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.238214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.265972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:01.277979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:01.279023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:01.279858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:01.287344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 211/424] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:01.296608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.347795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568ebad-39b1-4166-9963-6f81ed1e894e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.387813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b3da4-05c6-468d-85dc-f0d2b664abd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.389139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b3da4-05c6-468d-85dc-f0d2b664abd3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.389632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b3da4-05c6-468d-85dc-f0d2b664abd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.390112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b3da4-05c6-468d-85dc-f0d2b664abd3 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.391650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b3da4-05c6-468d-85dc-f0d2b664abd3 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.394372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b3da4-05c6-468d-85dc-f0d2b664abd3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.396732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 214/425] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:01.398498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcee8ab-d1b7-4c68-a842-c1fc485923ce None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.399001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcee8ab-d1b7-4c68-a842-c1fc485923ce None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.399527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcee8ab-d1b7-4c68-a842-c1fc485923ce None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.401326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 215/426] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:01.408720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcee8ab-d1b7-4c68-a842-c1fc485923ce None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.461232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.503810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea192c4-0fee-4526-95ce-9a04fe0565eb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.517954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.563278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcee8ab-d1b7-4c68-a842-c1fc485923ce None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.568162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.569317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.569814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.570303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.571645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:01.572206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.574709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfde55-d3b2-4a38-9920-24eebce2047c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.590976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 212/427] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:01.593136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80507236-c94f-45a9-8d61-a0c6da1bf97a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.593655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80507236-c94f-45a9-8d61-a0c6da1bf97a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.594156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80507236-c94f-45a9-8d61-a0c6da1bf97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.608105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.609136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.609545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.609959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.610971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=e59097a97ec144ff8463625775da1081, scope.project.id=d6ff45f877814dada97a096ebf743f75) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:01.611463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.613116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca250b-bdf7-4a2d-b650-53f92260dd01 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.620749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 216/428] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=e59097a97ec144ff8463625775da1081 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:01.622068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce45dd-30cf-410a-87bc-186219a94e01 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.622365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce45dd-30cf-410a-87bc-186219a94e01 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.622642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce45dd-30cf-410a-87bc-186219a94e01 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.624076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 217/429] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:01.628142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce45dd-30cf-410a-87bc-186219a94e01 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.631027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-80507236-c94f-45a9-8d61-a0c6da1bf97a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:01.633099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-80507236-c94f-45a9-8d61-a0c6da1bf97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:01.751270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 213/430] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:01.751270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.751270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.751270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.760366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.784178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:01.785820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:01.789214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce45dd-30cf-410a-87bc-186219a94e01 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.846955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.847664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.847960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.848244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.875450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 214/431] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:01.876335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.911590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.957659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.959783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.996252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456bfaa-60b9-4492-848b-4ae5b02a178d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.034448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8c43-6a5e-40cd-b6bb-8471625dff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.035228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8c43-6a5e-40cd-b6bb-8471625dff32 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.035510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8c43-6a5e-40cd-b6bb-8471625dff32 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.035788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8c43-6a5e-40cd-b6bb-8471625dff32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.046750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c8c43-6a5e-40cd-b6bb-8471625dff32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:02.054760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bb3c8c43-6a5e-40cd-b6bb-8471625dff32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:02.055777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:02.057555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:02.059127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:02.061934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 215/432] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:02.063620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a89a3c-14a6-44d7-9d82-910b335f6481 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.064781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a89a3c-14a6-44d7-9d82-910b335f6481 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.064781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a89a3c-14a6-44d7-9d82-910b335f6481 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.065939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 216/433] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:02.066675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a89a3c-14a6-44d7-9d82-910b335f6481 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.076103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 218/434] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:02.076954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.171298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.186908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a89a3c-14a6-44d7-9d82-910b335f6481 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.192974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffdf8a-6593-4852-9ea1-182ec4c70acc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.230519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.231374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.231657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.231934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.233832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97760902-3573-45e7-9543-9b37b415206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.234515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760902-3573-45e7-9543-9b37b415206e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.234797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760902-3573-45e7-9543-9b37b415206e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.235096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760902-3573-45e7-9543-9b37b415206e None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.236508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97760902-3573-45e7-9543-9b37b415206e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:02.236815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97760902-3573-45e7-9543-9b37b415206e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.238239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97760902-3573-45e7-9543-9b37b415206e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.247557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 219/435] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.248687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.248955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.249240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.250369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:02.267460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:02.268598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:02.269392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:02.270003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:02.270074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 217/436] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:02.270687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:02.271210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.320491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.347361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 220/437] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:02.355346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.411148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfed590-86eb-4788-8205-8680e0b3a80f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.448916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.449782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.450062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.450336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.451659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.453150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.454935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 218/438] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.455747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.467602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7da703-390e-44e5-a77f-fd32baae866f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.476387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.519803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.520478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.520766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.521039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.574678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6488ac-25ff-4e51-84f8-ffee106e4179 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.581652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.608173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.610297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.613325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.614118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.614470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.614808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.615610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:02.615913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.617316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.626022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 219/439] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.627681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.724489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.726357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 221/440] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:02.727210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.749168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889a91-ab3f-4acd-ac50-fbb906e8d019 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.788010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.789213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.789642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.790070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.807891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:02.822452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:02.823628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:02.824428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 220/441] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:02.825682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.827250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.853043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cc919-65a5-4de7-866c-db04ed6cb7ba None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.891007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.892767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.893223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.893688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.895242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.897258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.907449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 222/442] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.907449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.949245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b305f0-b309-4283-adbf-3aefa132b2c2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.987401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.988061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.988346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.988631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.999309 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.018865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e524e-4bc5-4884-b863-af0b988be4b5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.055408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.056434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.056725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.056992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] PATH_INFO: `/v3/projects/3be9f13173874417a10d07a7e522883a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.058261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] RBAC: Authorizing `identity:get_project(project_id=3be9f13173874417a10d07a7e522883a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.059753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.061487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 223/443] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/projects/3be9f13173874417a10d07a7e522883a => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.062266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.064979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.067932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.080173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.174587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173234ea-5030-41a9-b0f8-c357e1590395 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.185346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 221/444] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:03.186243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.219368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.220614 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.221009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.221400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.222384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:03.222676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.224135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.232241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.233161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 224/445] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.234019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.298781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4cfa-a020-4a4d-844a-486f5f8830f0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.321656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.336515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.337098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.337380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.337651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.338499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=e59097a97ec144ff8463625775da1081, scope.project.id=3be9f13173874417a10d07a7e522883a) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:03.338813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.340312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.347132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 222/446] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=e59097a97ec144ff8463625775da1081 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.347681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcde02-f1a1-487b-abde-e356abb053b7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.347969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.411896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.412997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.413366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.413747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.415532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.417421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.420866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 225/447] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.425430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.530658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97740adc-fa99-45cd-9519-6b3b8d7132e2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.533418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.585894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d331-0301-446a-b89f-5b625d1a1f3c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.600715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.601742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.602198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.603353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] PATH_INFO: `/v3/projects/d6ff45f877814dada97a096ebf743f75` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.604489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff45f877814dada97a096ebf743f75)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.606980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.614069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 223/448] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/projects/d6ff45f877814dada97a096ebf743f75 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.614069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.659212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.660208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.660708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.661176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.690244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.786552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab902d-56a4-4268-a92f-a8c98b3a2152 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.793752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.795613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.842483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.843273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.843518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.843792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.844634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=852d973419ee442c80da4214c3df867b, scope.project.id=3be9f13173874417a10d07a7e522883a) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:03.844945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.847044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.853800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 224/449] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=3be9f13173874417a10d07a7e522883a&role.id=852d973419ee442c80da4214c3df867b => generated 611 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.854562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.912028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.949947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 226/450] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:03.956816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.011624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.033075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a88746-3cdf-498e-87f9-b90deb21cb83 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.088326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.089263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.089534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.089837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] PATH_INFO: `/v3/groups/515033a21229473a9ebee545c0cde34e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.091225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] RBAC: Authorizing `identity:get_group(group_id=515033a21229473a9ebee545c0cde34e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.092721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.094432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 225/451] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/groups/515033a21229473a9ebee545c0cde34e => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.099845 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e5259-396b-47ae-ab7f-bfa05c297bcd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.102487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.146869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.147702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.147979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.148502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] PATH_INFO: `/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.149769 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] RBAC: Authorizing `identity:get_project(project_id=1a2e5fa7e325464b9d8cfafd39af8ec6)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.151284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.153103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 227/452] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/projects/1a2e5fa7e325464b9d8cfafd39af8ec6 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.159105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.266869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f9c65-b481-49f3-a1cc-d58c21f8813e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.269525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.324936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6557ba-9ec1-48a6-8383-347180334231 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.333036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.333728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.334010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.334956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.335917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] RBAC: Adding query filter params (group.id=515033a21229473a9ebee545c0cde34e, role.id=852d973419ee442c80da4214c3df867b, scope.project.id=d6ff45f877814dada97a096ebf743f75) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:04.336238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.338370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.343750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.347555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 226/453] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/role_assignments?group.id=515033a21229473a9ebee545c0cde34e&scope.project.id=d6ff45f877814dada97a096ebf743f75&role.id=852d973419ee442c80da4214c3df867b => generated 611 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:04.392730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4774c4-8c87-40b1-9cdf-e3d68bc28bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.393136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4774c4-8c87-40b1-9cdf-e3d68bc28bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.393423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4774c4-8c87-40b1-9cdf-e3d68bc28bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.393901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4774c4-8c87-40b1-9cdf-e3d68bc28bf8 None admin] PATH_INFO: `/v3/groups/b0ea0594926440aa9628cb1924af59f8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.396078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4774c4-8c87-40b1-9cdf-e3d68bc28bf8 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ea0594926440aa9628cb1924af59f8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.398669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4774c4-8c87-40b1-9cdf-e3d68bc28bf8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.401336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 228/454] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/groups/b0ea0594926440aa9628cb1924af59f8 => generated 229 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:04.407879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.562756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfdd29-38bc-4f83-a6a4-1f8051b01345 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.600125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.600795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.601160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.601452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.602347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] RBAC: Adding query filter params (group.id=b0ea0594926440aa9628cb1924af59f8, role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=1a2e5fa7e325464b9d8cfafd39af8ec6) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:04.602662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.604217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449290bd-a010-4320-abd7-2cb3118f3362 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.611660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 227/455] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/role_assignments?group.id=b0ea0594926440aa9628cb1924af59f8&scope.project.id=1a2e5fa7e325464b9d8cfafd39af8ec6&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:07.230338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf61d85-af55-4311-a884-a8119549fb5b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.231182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf61d85-af55-4311-a884-a8119549fb5b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.231182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf61d85-af55-4311-a884-a8119549fb5b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.232696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 229/456] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:07.239080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786b6f2-f34d-47ca-ab84-adf22b836ba2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.240050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786b6f2-f34d-47ca-ab84-adf22b836ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.240483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786b6f2-f34d-47ca-ab84-adf22b836ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.262581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2786b6f2-f34d-47ca-ab84-adf22b836ba2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:07.264119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2786b6f2-f34d-47ca-ab84-adf22b836ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:07.303099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:07.366217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 228/457] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:07.413288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf61d85-af55-4311-a884-a8119549fb5b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:07.489718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da45c-a7fe-479a-b71d-d51093c8a844 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.490344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da45c-a7fe-479a-b71d-d51093c8a844 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.490923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da45c-a7fe-479a-b71d-d51093c8a844 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.492864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 229/458] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:07.498643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.499277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.499766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.526979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:07.529034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:07.745825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf61d85-af55-4311-a884-a8119549fb5b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:07.756082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 230/459] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:07.797227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:07.824105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74741-c449-496e-b486-29163327febe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:07.825370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74741-c449-496e-b486-29163327febe None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.825839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74741-c449-496e-b486-29163327febe None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.826300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74741-c449-496e-b486-29163327febe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.853653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74741-c449-496e-b486-29163327febe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:07.859109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74741-c449-496e-b486-29163327febe None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-77c74741-c449-496e-b486-29163327febe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:07.911684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:07.913326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:07.914842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:07.914842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:07.914842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:07.914842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:07.920036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 230/460] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:07.934946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74741-c449-496e-b486-29163327febe None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:07.954875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:07.995585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073b909-3321-42f5-a42c-1b0d528fdcae None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.015773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:08.134329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.136089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.136779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.137477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.165177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:08.168283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:08.206624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:08.208156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:08.209660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:08.209660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:08.209660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:08.209660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:08.213359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 231/461] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:08.214269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74741-c449-496e-b486-29163327febe None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.227869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.259941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.260767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.260889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.261195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.261891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:08.262200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:08.263652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:08.277683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 231/462] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:08.284196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.443045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fe1dc-6a84-43b7-9e96-05deba57e67e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.500713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:08.515997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.517366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.517725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.518024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.518801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:08.519154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:08.520657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:08.531500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f7593-97ac-4289-b416-945fe063aa0a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.547317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 232/463] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:08.552774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.555728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:08.611073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d340f-f418-414e-b5de-dc5bcf038078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.612184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d340f-f418-414e-b5de-dc5bcf038078 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.612184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d340f-f418-414e-b5de-dc5bcf038078 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.612472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d340f-f418-414e-b5de-dc5bcf038078 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.613249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d340f-f418-414e-b5de-dc5bcf038078 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:08.614891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d340f-f418-414e-b5de-dc5bcf038078 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:08.650457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 232/464] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:08 2022] POST /identity/v3/users => generated 246 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:08.651418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11d357-5257-4da7-b865-d2b97963aae9 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.651772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11d357-5257-4da7-b865-d2b97963aae9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.652056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11d357-5257-4da7-b865-d2b97963aae9 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.659193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 233/465] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:08.669763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.670052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.670330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.737768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:08.740293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:08.783989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ae28e-0995-48a5-ab85-23dfcd4d64d1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.855096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.873524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.873524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.873524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.873524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:08.873524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:08.890582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 233/466] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:08 2022] POST /identity/v3/users => generated 248 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:08.927284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 234/467] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:08.948846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.026465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.078708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ab5a4-f47c-4a4e-8db5-a4eff7df4526 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.124221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.125491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.126087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.126604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.140539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.143242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-75084a2f-e21d-4f8f-86bc-9e5059161207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:09.165581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:09.166673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:09.167615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:09.167615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:09.167615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:09.167615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:09.170691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 234/468] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:09.174866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.214867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118335-db15-4256-8f98-09b3738a5254 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.215344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118335-db15-4256-8f98-09b3738a5254 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.215729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118335-db15-4256-8f98-09b3738a5254 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.217914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 235/469] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:09.226810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.227487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.227895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.264435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:09.267789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:09.340044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.370195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25bc9-f76c-4486-9b9a-825a3faf362e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.408745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 236/470] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:09.431159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.432322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.432805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.433295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.434669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:09.435241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.437849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.447532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.462179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 235/471] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:09.473039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.526789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.645698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65600c7c-530d-41b7-a51f-370040c7661f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.705373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.706510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.706990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.707505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.721796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.723803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:09.754977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:09.755928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:09.756772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:09.756772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:09.756772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:09.756772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:09.763649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 237/472] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:09.780021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0c896-3aeb-4be1-956a-ca4e81fe15fb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.785536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.874644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73642d-41b9-4678-9b80-290fa41a4a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.876728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73642d-41b9-4678-9b80-290fa41a4a24 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.877255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73642d-41b9-4678-9b80-290fa41a4a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.877760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73642d-41b9-4678-9b80-290fa41a4a24 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.880118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73642d-41b9-4678-9b80-290fa41a4a24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.884018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73642d-41b9-4678-9b80-290fa41a4a24 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.903446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.928936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.933587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 236/473] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:09 2022] POST /identity/v3/users => generated 248 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:09.937584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966405-cfe5-4a52-a8c0-7a894d90996c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.939339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966405-cfe5-4a52-a8c0-7a894d90996c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.939652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966405-cfe5-4a52-a8c0-7a894d90996c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.947193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 237/474] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:09.950861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.951906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.952677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.957902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31fb99-7814-4936-8aa0-f17d05705653 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.989155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:09.991326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:10.043887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.044574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.044859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.045191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.045942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:10.046249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.048441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.077584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 238/475] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:10.087691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.134521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 238/476] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:10.171993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.309104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61942dd0-deb1-4bec-a385-362197927f29 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.371403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9428dee-0307-4cce-8c6a-3a3ad761cc9e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.384383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.385082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.385369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.385642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.386363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.387956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.414695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 239/477] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:10 2022] POST /identity/v3/users => generated 249 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:10.422123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.437560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.438337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.439401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.439676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.440606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.442075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.451213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.sql.core [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:19:10.451213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 22:19:10.451213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [parameters: {'id': 'ae115306bcc04998bac7ea1f0c2ab55c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 22:19:10.451213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80276) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 21 22:19:10.471017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:19:10.472762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application yield resource Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise self.value Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application session.commit() Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:19:10.474585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:19:10.476341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:19:10.478064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:19:10.482537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [parameters: {'id': 'ae115306bcc04998bac7ea1f0c2ab55c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:10.483376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 22:19:10.484298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:10.485161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 239/478] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jul 21 22:19:10 2022] POST /identity/v3/roles => generated 144 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:10.485331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.520674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.742215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0a6ca-9794-44e8-921c-999c51daca90 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.788220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22edf9-b469-4c55-b41d-7d8b3ccfbefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.789369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22edf9-b469-4c55-b41d-7d8b3ccfbefb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.789831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22edf9-b469-4c55-b41d-7d8b3ccfbefb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.790272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22edf9-b469-4c55-b41d-7d8b3ccfbefb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.803512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22edf9-b469-4c55-b41d-7d8b3ccfbefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6a22edf9-b469-4c55-b41d-7d8b3ccfbefb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:10.812464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:10.814320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:10.815297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:10.817713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 240/479] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:10.826916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.973134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dc883-f79c-47f1-93c9-b1247f664dd0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:11.041127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:11.042171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.042657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.043174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.044590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:11.045207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:11.048449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef612ee8-c9b9-4e43-b277-17e1c04f1836 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:11.072594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 240/480] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:12.874939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac92c9d-bdf0-45ba-9c3f-a1b4676a19e8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.876756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac92c9d-bdf0-45ba-9c3f-a1b4676a19e8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.877282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac92c9d-bdf0-45ba-9c3f-a1b4676a19e8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.879384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 241/481] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:12.886963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f2f1f-2c01-4956-afa9-b2645865a4bc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.888585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f2f1f-2c01-4956-afa9-b2645865a4bc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.889392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f2f1f-2c01-4956-afa9-b2645865a4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.920287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-388f2f1f-2c01-4956-afa9-b2645865a4bc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:12.921941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-388f2f1f-2c01-4956-afa9-b2645865a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:12.991488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.026836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 241/482] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:13.066983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac92c9d-bdf0-45ba-9c3f-a1b4676a19e8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.121863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00271091-9a73-4eba-9ff0-beea003fe4e9 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.122321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00271091-9a73-4eba-9ff0-beea003fe4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.122763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00271091-9a73-4eba-9ff0-beea003fe4e9 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.125397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 242/483] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:13.135596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fd015-61ea-482f-a21e-d850eb8fc0da None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.136077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fd015-61ea-482f-a21e-d850eb8fc0da None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.136555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fd015-61ea-482f-a21e-d850eb8fc0da None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.140594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 243/484] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:13.143040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b189-a2a8-42b8-8561-cb89fce4cdd6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.146456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b189-a2a8-42b8-8561-cb89fce4cdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.147121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b189-a2a8-42b8-8561-cb89fce4cdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.179980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9a65b189-a2a8-42b8-8561-cb89fce4cdd6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:13.181821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9a65b189-a2a8-42b8-8561-cb89fce4cdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:13.191312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.239357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac92c9d-bdf0-45ba-9c3f-a1b4676a19e8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.283954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.285192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.285661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.286131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.299148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:13.307709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:13.308774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:13.309562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:13.311844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 244/485] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:13.317395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.317395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.317395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.318009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 242/486] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:13.319285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.347741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:13.350278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:13.475691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717973-a7f0-484e-9462-36bd588e2668 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.498577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 245/487] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:13.499832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.546518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.554811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.569872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.570941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.571467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.572148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.573194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:13.573821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:13.575602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:13.586194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 243/488] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:13.587442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.682143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f78629-ddfb-43b4-90b8-f93cb5c64869 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.735192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.737233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.737758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.738260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.753448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:13.762338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:13.763409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:13.764293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:13.772812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 246/489] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:13.774426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.802384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9193-2cf4-459c-966f-ded7c46049be None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.863011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.864411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.865375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.865924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.884660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:13.901576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:13.902606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:13.903436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:13.907780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 244/490] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:13.909509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.956692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.961965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde01998-b29d-4ff7-a34a-d6ce6dd7cdbb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.027237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.029026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.030480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.030974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.031498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.054638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.057430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:14.097644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:14.099707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:14.101410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:14.101410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:14.101410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:14.101410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:14.101410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 247/491] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:14.102192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.137541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0b1b-73f8-42de-8400-b5d5f870e4dd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.204688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.206239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.206975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.207778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.209423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:14.210229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.213136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.229825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 245/492] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:14.231868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.305391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a14088-2330-4307-b222-5c66e47f7463 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.373624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.374771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.375308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.376370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.377965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:14.378552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.381238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.405379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 248/493] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:14.407671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.432491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a08e6-f2b9-4aa6-959b-7d5f31a0697c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.443548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.467742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.499377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.500582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.501264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.501548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.502803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:14.503159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.504596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.523130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 246/494] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:14.526593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.622229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fb2c9-e59e-44ed-99cd-d430f5c40fb0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.692978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.693671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72076cd0-8580-4496-8093-7dc21701a5a7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.694326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.694823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.695346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.717174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.721123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:14.758896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:14.760631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:14.762273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:14.762273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:14.762273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:14.762273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:14.762273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 249/495] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:14.762804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.767681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.769623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.770413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.771223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.792754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.796467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:14.837360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:14.838992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:14.840671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:14.840671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:14.840671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:14.840671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:14.845040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 247/496] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:14.846905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.904703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.915551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.979174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec15df3-df2a-4b72-91cc-6c284c95d2a8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.037454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.038272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.038561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.038843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.040108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d07745-c001-401d-b4ea-d59c606adbaf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.048313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.049805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.077411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.078077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:15.078222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:15.080679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:15.088613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:15.090016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.090016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.090016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:15.090016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.090748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 250/497] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:15.090888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2d436-b935-42f9-b93e-5806bb19f860 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.091043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.096706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 248/498] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:15.098170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd3e8-6c9d-4a84-b89f-15d71f59ec9e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.098526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd3e8-6c9d-4a84-b89f-15d71f59ec9e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.098860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd3e8-6c9d-4a84-b89f-15d71f59ec9e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.100542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 249/499] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:15.101573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492805a-05ce-4219-92cb-5cf866e4d24c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.101857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492805a-05ce-4219-92cb-5cf866e4d24c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.102237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492805a-05ce-4219-92cb-5cf866e4d24c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.104005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 250/500] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:15.104537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492805a-05ce-4219-92cb-5cf866e4d24c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.226063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.242525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c96e-06cd-447a-a319-49286c1afee0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.254550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492805a-05ce-4219-92cb-5cf866e4d24c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.278158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.290180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.290943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.291319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.291595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.292336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:15.292644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.294054 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.299800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.300965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.301469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.301957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.303655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:15.304205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.307034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef45a1-7d11-4b3e-8e2f-3812d5c210ee None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.307950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 251/501] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:15.309055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.384638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 251/502] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:15.386985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.387532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.388032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.419395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:15.421437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:15.432208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7e404-3fec-4b5c-84fb-32bb4f0b74b4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.471185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.471990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.472266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.472534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.481037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.482439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9d4a1c50-3e33-4a39-99c8-c70d328c01eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:15.498486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:15.500378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:15.503179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:15.506079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 252/503] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:15.508237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.508729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.509030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.527734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:15.528965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:15.562634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 252/504] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:15.562634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.589557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.635093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 253/505] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:15.635093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.657826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.710019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf809-9a45-4385-a45c-2a0066c91e13 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.757843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9768fdc-e3cf-4cbe-9a15-3ad31925e3b0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.772752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.773983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.774431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.774763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.784667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.786316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.803020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.803898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.804238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.804600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:15.805751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:15.807644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 253/506] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:15.816996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.884762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.920120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.921915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.960194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77af663-40d2-46fe-8898-6663b50b51e3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.996334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.002449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.003166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.003454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.003727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.004576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:16.004885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.006354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.026905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 254/507] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:16.027696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.045461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 254/508] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:16.046325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.142454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.167363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc6db3-b889-4356-8026-d0ae265f4e60 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.172187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d295de-9463-4894-9b3b-94e9efafdd8c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.211072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.212261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.212713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.213157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.218804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.220652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.221093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.221536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.229262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:16.237397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:16.239039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 255/509] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:16.240619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.244461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.247492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:16.276842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:16.278576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:16.280785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:16.280785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:16.280785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:16.280785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:16.280785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 255/510] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:16.280785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.312002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.384381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc55566-7270-4c98-a943-b7040aa4c1e8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.404367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6553985-e367-40b7-8d0e-30a6269100ac None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.420957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.421635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.421911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.422181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.423970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:16.424292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.425770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.445391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 256/511] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:16.446116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.456781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.467662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.468107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.468451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.468790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.565748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.568338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.577357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02602275-7ea9-4f92-a1b6-d13299626f49 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.595229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.628236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.629046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.629327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.629603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.641560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.643504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:16.663672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:16.665263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:16.667543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:16.667543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:16.667543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:16.667543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:16.667543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 257/512] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:16.667543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.722522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 256/513] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:16.724081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.774895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.784231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b49a-13f1-4b9a-9d50-f8bfb3a5de3d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.826512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.827241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.827591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.828001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.829668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:16.830208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.831733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.841534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 258/514] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:16.842399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.875924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b6299-a6f9-400b-9378-10aa1cd78510 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.880566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.925161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.925855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.926135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.926512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.927366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:16.927809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.929392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.944431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 257/515] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:16.945304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.998057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4014ff9-e98c-420f-badc-47c9a127e448 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.048672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.057876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.058791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.059221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.059555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.066644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c9744-a3fc-4be0-b45c-46986239da25 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.117393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.118403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.118781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.119189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.120242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:17.120736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.122286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.136989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 258/516] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:17.138080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.151970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.153721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.182399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.262686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542a4d-788a-43a3-97d8-4a5502607346 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.277024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 259/517] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:17.277885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.326394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.327822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.328340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.329987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.348714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.351568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.368029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.387687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225fbeb-acfa-476c-9c5f-9a2660f4961b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:17.387976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:17.389537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:17.390637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:17.390637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:17.390637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:17.390637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:17.390637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 259/518] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:17.392419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.424069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.436706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.437520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.437787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.438332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.450409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.451920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:17.472297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:17.474037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:17.476008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:17.476008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:17.476008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:17.476008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:17.476008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 260/519] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:17.477676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.578433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43dd2a-4df7-450f-ba01-41bfc802f4e9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.597750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf392d-c332-49e6-85ac-02929021aa61 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.633904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.634992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.635263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.635518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.635965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.636097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.636337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.636609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.637285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.640144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.644872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.646419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:17.663408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:17.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 261/520] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:17.666497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.682395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 260/521] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:17 2022] POST /identity/v3/users => generated 247 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:17.683951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.728671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.764794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.818878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1325c5d-7a1f-49a0-ac3e-4ecc9ed41ce0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.852831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb99266-e394-4818-aa27-9742fac2ba64 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.897089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.899614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.899994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.900359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.907815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.908968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.909462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.909959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.911363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:17.911909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.914496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.918316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.920866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.934058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 261/522] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:17.935578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:17.943408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:17.944368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:17.945651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:17.945651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:17.945651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:17.945651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:17.945651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 262/523] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:17.945651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.084989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.103593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d114-303d-4bee-9e1d-d77a4d37a737 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.107899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97961d-e847-4696-9e52-75ab8e9744b0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.136214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.150487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.151246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.151610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.152175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.153524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:18.153640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.155085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.172894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.173895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.174309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.174712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.175895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:18.176360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.179412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.182509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 263/524] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.183193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.211027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 262/525] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.211027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.326322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125596-0577-4dcd-a69d-290b99ec3fa6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.340915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd6fc6-5b4e-46db-8eb0-a9abef8dc233 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.369644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.370468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.370835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.371033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.379940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.381983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.395718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.396571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.397000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.397430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.398584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:18.399027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:18.399680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:18.401258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:18.403098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:18.403890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.406051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 264/526] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:18.407861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.427213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 263/527] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.427213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.427675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.489782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.569509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbe21d-ed3b-496e-a13a-4fe236f55639 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.609050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b8ab5-98f3-49b7-a4a1-7788dfef10b6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.633196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.633873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.634154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.634422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.635296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=054c95ae7aa645b7ade6390e9bf0059f) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:18.635612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.637110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.647290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 265/528] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/role_assignments?user.id=054c95ae7aa645b7ade6390e9bf0059f&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.650689 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.675435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.678021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.678583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.679790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.697646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.701281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:18.735688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:18.736771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:18.737706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:18.739275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 264/529] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:18.740531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.773766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.819637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5d493-cf68-4200-be6e-b10ab17530bc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.888698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.890195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.892275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.893006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.893114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.894169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:18.894499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.896041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.910508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fd96a-b436-423c-83b4-5e94b9383c80 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.917512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 266/530] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.918437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.005771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.008192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.008564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.008795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.028113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.033878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.049832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5612d6-fc72-4adf-a250-5672b1eff803 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:19.074316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:19.075731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:19.077248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:19.084703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 265/531] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:19.091362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.118481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:19.129567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.131387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.132426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.133134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.134893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:19.135634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.138531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.176029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 267/532] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.177551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.335855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf2f12-36f0-473f-b9c4-cf51e0c3c5a9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.396649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847de912-6725-4acb-b3f0-bdf846caabce None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.424157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.425363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.425986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.426672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.428302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=d529f6af558a44ef909937d5a960cb88) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:19.429413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.432278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.446318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 266/533] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/role_assignments?user.id=d529f6af558a44ef909937d5a960cb88&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.448207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.491232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.492615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.494003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.494492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.496766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:19.497335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.501633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.529256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:19.532470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 268/534] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.544282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.647592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:19.677903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcdb4a-4de4-4b1a-8ab3-23394ad6dc65 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.744211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f10fb-397a-4bf1-bf7d-e9f25aa893f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.745471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f10fb-397a-4bf1-bf7d-e9f25aa893f6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.746051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f10fb-397a-4bf1-bf7d-e9f25aa893f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.746544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f10fb-397a-4bf1-bf7d-e9f25aa893f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.791037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226cf95-0f24-4a05-a065-f456cf86c448 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.875938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.876893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.877327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.877767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.878760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=3df4115fcce342f2ae095ec1a2af4cca) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:19.879260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.880879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.887547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 269/535] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/role_assignments?user.id=3df4115fcce342f2ae095ec1a2af4cca&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.956227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f10fb-397a-4bf1-bf7d-e9f25aa893f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.958988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f10fb-397a-4bf1-bf7d-e9f25aa893f6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:20.001122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:20.189663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 267/536] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:20.197297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:20.394841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99c1ae-e05a-4b04-8bb9-0c9a9605c212 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:20.420409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:20.463902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2dc7-3d5f-45b9-a907-1509f30863a6 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.464457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2dc7-3d5f-45b9-a907-1509f30863a6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.464821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2dc7-3d5f-45b9-a907-1509f30863a6 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.467342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 268/537] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:20.476272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.476610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.476822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.481341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:20.482709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.483260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.483767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.505119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:20.507952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:20.511560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:20.512776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:20.536401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:20.538108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:20.541184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:20.541184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:20.541184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:20.541184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:20.541184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 270/538] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:20.546533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:20.703034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 269/539] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:20.790429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6675e-f7f1-47ab-996b-99459d5b06f6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:20.817616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:20.880682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:20.881751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.882256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.882732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.883700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:20.884233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:20.886176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:20.914198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 271/540] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:20.922356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:20.955930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:20.979200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:21.067963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7848a-fd24-4138-bb41-7ea5e742b618 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:21.130422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a36d0-cac5-40f1-9cd2-c05a6d3a4820 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:21.161598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:21.162841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.163344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.163800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.183668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:21.184832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.185319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.185794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.197597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:21.206560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:21.207522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:21.208340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:21.211487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 270/541] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:21.215123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:21.215123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:21.220995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a53155d7-4a56-4ba1-8292-b34fadc4ef52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:21.245242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:21.247025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:21.248823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:21.251086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 272/542] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:21.253319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23dc93-7fb5-4d41-a1d9-6c25b5ac6e0a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.253792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23dc93-7fb5-4d41-a1d9-6c25b5ac6e0a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.254248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23dc93-7fb5-4d41-a1d9-6c25b5ac6e0a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.257941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 273/543] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:21.260816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23dc93-7fb5-4d41-a1d9-6c25b5ac6e0a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:21.394026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03d4d8-6524-4fbf-88f1-628aa76abee2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:21.399984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:21.418972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23dc93-7fb5-4d41-a1d9-6c25b5ac6e0a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:21.419236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:21.457549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:21.458449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.458880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.459345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.460320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:21.460789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:21.462398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b9de-81ff-4f59-8730-368706a2a673 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:21.468527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:21.470201 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.470701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.471224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.472780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:21.473339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:21.476110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:21.482969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 274/544] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:21.485266 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.485947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.486527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.493636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 271/545] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:21.495257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:21.505355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:21.506819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:21.581477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 275/546] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:21.582752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:21.668885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:21.697496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc79489-e82a-4314-a6ea-c53c2a49cce7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:21.702079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674f5fb-ad3d-4bab-95ab-44a3fdf5e6b0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:21.737806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:21.738844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.739328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.739763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.752930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:21.754730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:21.774031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:21.775341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:21.775543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:21.776431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:21.777430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:21.777430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:21.777430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:21.777430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:21.777664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.777664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.777944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=7f5dceea71d1434796a105ea67773cd2) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:21.778491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:21.780021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 276/547] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:21.781179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:21.781476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:21.794049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 272/548] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:21 2022] GET /identity/v3/role_assignments?user.id=7f5dceea71d1434796a105ea67773cd2&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:21.795604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:21.872019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.018155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92bf28-016d-4f3d-8dec-a9e601ab5174 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.051038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.061610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf3e2c-7456-4125-a691-1e74f99f660c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.063232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.064094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.064390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.064643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.083500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:22.092617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:22.094436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:22.096587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 277/549] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:22.101491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.164140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.164824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.165109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.165425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.166173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:22.166479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.167912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.196401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 273/550] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:22.202132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.319072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.320957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1605-32cc-4008-a912-2e832da3e34a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.322025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a9bc-47e2-475a-b573-3c760474231e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.363621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.364744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.365252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.365736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.366731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:22.367310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.369905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.379331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.379952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.380230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.380603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.383806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 278/551] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:22.392831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.394865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.396615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:22.426581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:22.427741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:22.428671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 274/552] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:22.439321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.504822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.646374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d819-db70-432d-a445-60f73ba6d2f4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.672434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a625f72-a142-4b5e-9593-ea68041d9e67 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.711453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.713264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.713764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.714255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.738395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.740718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:22.773489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:22.775077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:22.778374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:22.778374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:22.778374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:22.778374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:22.778816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.778816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.779102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 279/553] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:22.779230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.779230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.779230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:22.779230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.779230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c343543-d5d0-4c8b-a938-d503d55cef61 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.785456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.842241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 275/554] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:22.849186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547919f-b2e3-459c-9b3c-9ab288ac8214 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.849422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547919f-b2e3-459c-9b3c-9ab288ac8214 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.849688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547919f-b2e3-459c-9b3c-9ab288ac8214 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.858490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 276/555] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:22.858814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547919f-b2e3-459c-9b3c-9ab288ac8214 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.879765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.950143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.957413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5947fbe-ad13-4451-8840-506917ad775a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.018100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547919f-b2e3-459c-9b3c-9ab288ac8214 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.023935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.025638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.028864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.029916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.032104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:23.033426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.036394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4572ee-7afe-46b3-8451-f98792030978 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.061944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 280/556] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:23.064919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440692a-92d3-4a55-bd13-75ca3c4777f6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.065305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440692a-92d3-4a55-bd13-75ca3c4777f6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.065597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440692a-92d3-4a55-bd13-75ca3c4777f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.093511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fedbe-c3ec-424b-9ea0-26ce1fbf5ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.093511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fedbe-c3ec-424b-9ea0-26ce1fbf5ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.093511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fedbe-c3ec-424b-9ea0-26ce1fbf5ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.093511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fedbe-c3ec-424b-9ea0-26ce1fbf5ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.100963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1440692a-92d3-4a55-bd13-75ca3c4777f6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:23.105291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1440692a-92d3-4a55-bd13-75ca3c4777f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:23.201182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fedbe-c3ec-424b-9ea0-26ce1fbf5ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.203741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fedbe-c3ec-424b-9ea0-26ce1fbf5ed8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.225776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 281/557] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:23.227663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09474b8-32fa-4742-9dc3-28e4e4046c1f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.228181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09474b8-32fa-4742-9dc3-28e4e4046c1f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.228693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09474b8-32fa-4742-9dc3-28e4e4046c1f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.230636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 282/558] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:23.231898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09474b8-32fa-4742-9dc3-28e4e4046c1f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.239020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.355500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.370165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09474b8-32fa-4742-9dc3-28e4e4046c1f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.396042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 277/559] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:23.397321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.398071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.398987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.423992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:23.425305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:23.430288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.431441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.431906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.432389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.442873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.444831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:23.464125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:23.465870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 283/560] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:23.467750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.551596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 278/561] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:23.552950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.645125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.657655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c857b0-60d0-4a67-9309-31cb9d03e84e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.733292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.733890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.734172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.734899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.741260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690bdd-2855-4b3b-9671-ea6be9da649a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.748450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:23.757609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:23.759712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:23.764547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 284/562] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:23.765987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.781165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.821982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.823414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.823931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.824427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.846120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.848942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:23.882367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:23.883744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:23.884656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:23.884656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:23.884656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:23.884656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:23.885437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 279/563] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:23.886574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.907654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fb24f-b68f-4f58-a11a-d2da1012360c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.965929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.967153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.967606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.967893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.970195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:23.970686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.973507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.007688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 285/564] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.007688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.074005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.085839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0e911-1abf-473e-95a5-1139f74846fd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.128925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.129759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.130050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.131035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.132189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.143447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:24.152310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:24.153549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 280/565] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:24.155211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.210426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fbe7-0269-4c38-ba93-4c94a444eee8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.278091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.279114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.279597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.280075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.281415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:24.281934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.284340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.298656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 286/566] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.299763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.309360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82376dc2-0993-4273-94b3-3188140abd47 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.370727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.371642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.372062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.373145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.373727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:24.374181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.376544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.399933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 281/567] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.399933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.424603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.480673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccff41-605a-4ab2-a0a7-9dc6d3b00518 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.535215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.546558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.548490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.548997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.549472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.570277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.573352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.576596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70175ed-caed-4f46-87d2-6cc42ae7af3b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:24.597112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:24.599091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:24.601294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:24.601294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:24.601294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:24.601294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:24.601294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 287/568] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:24.601294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.627312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.628111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.628466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.628809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.629819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:24.630196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.631816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.643096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 282/569] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.643793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.754078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.772130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d8d1d-2981-4ef0-b01d-04c8ab25e942 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.789641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978447a-016a-45f0-90f5-8b80f643beed None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.824650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.825795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.826259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.826702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.841864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.843319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.843935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.844452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.844895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.847587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.856832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.859786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:24.875922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:24.877888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:24.880137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:24.880137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:24.880137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:24.880137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-0dfcacb0-83e7-4771-bfd8-f8afad87f774 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:24.880755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:24.883041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:24.885161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 283/570] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:24.886398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 288/571] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:24.886398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.886671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b3dd-166b-4315-a37e-ff9afa418672 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.886671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b3dd-166b-4315-a37e-ff9afa418672 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.886671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b3dd-166b-4315-a37e-ff9afa418672 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.887463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 284/572] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:24.888487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013b3dd-166b-4315-a37e-ff9afa418672 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.932850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.021430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d21b23-0a63-4914-baf2-050172f00eb1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.030704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013b3dd-166b-4315-a37e-ff9afa418672 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.070927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.071895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.072263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.072456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.072718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.073441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.073945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.074447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.075875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:25.076431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.078969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.080824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.085837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.087354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.109241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 289/573] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:25.109241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:25.111407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:25.113037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:25.116753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:25.116753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:25.116753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:25.116753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:25.116753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 285/574] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:25.116753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.214129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.222668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a278fb6-d0f0-4409-bbbe-4a7855152c52 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.251608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51fac6-c69b-4d88-b422-7adccc749a73 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.261404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.262564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.263082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.263602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.264991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:25.265615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.268275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777fe0a-5a7f-4b74-ab8b-cd45561b6ba5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.293586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 290/575] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:25.293779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.293875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.293875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.294036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.294036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.294159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.294227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.294227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:25.294227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.295270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.314930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 286/576] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:25.315853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.318920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:25.321138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:25.375716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.405014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 291/577] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:25.405880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.444467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87e0e7-20df-4b8d-b057-0785dd5ca238 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.483486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.484789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.485070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.485341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.492642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.494101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:25.509168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:25.510944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 287/578] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:25.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.520920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.541758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919309a-e59a-4a60-a061-136e618bef2a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.578497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.579357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.579645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.580035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.580774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:25.581081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.582478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.595234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 292/579] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:25.596463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.655937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650f80a-0aa9-45dc-82a6-8ab6d9311ed8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.669334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.697680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.698601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.698787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.699103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.709196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.710719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.721886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c12ec2-d3e8-47a9-81f4-8df5ba053579 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:25.726402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:25.728404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 288/580] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:25.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.766764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.767608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.767895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.768178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.769042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=08ed51a3aab34ac0bf5791fa3307a79d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:25.769366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.770884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.777707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 293/581] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/role_assignments?user.id=08ed51a3aab34ac0bf5791fa3307a79d&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:25.778582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.798841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.887582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaedee-a02e-4fac-bda4-8a8ccd439771 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.938578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42fd07-d046-4e6b-bf9c-d3eaa5dc4872 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.949080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.950319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.950762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.951248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.971112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:25.985596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:25.986641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:25.987789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:25.988442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 289/582] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:25.989301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.008200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.009386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.009680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.009957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.010766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:26.011676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.012625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.027572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 294/583] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:26.028451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.049358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.139203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1ebf-d0cb-46b1-a0a3-99e83160ff8c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.168893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.188467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.189342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.189631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.189896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.198768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.200305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:26.220295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:26.221348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:26.222299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:26.222832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 290/584] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:26.224002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.245064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d31bf-615e-40df-aa5e-475614b89e5f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.309342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.310742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.311304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.311792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.313923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:26.314949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.317237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.346723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.351967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 295/585] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:26.353423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.364523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d93c3b-36c5-4893-90f2-4b1cf2dc4918 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.404149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.404762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.405042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.405322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.407048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:26.407459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.408854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.419101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 291/586] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:26.419546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.516631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ac51-e98f-4fac-9205-02cabb08f5bb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.536329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556fd2-31dc-4c95-9186-135d34b99f61 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.546169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.557096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.558002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.558278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.558629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/054c95ae7aa645b7ade6390e9bf0059f/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.577332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=054c95ae7aa645b7ade6390e9bf0059f, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.579001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.580112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.580806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.581109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.581600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.582902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:26.583259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.584770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.603098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 296/587] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:26 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/054c95ae7aa645b7ade6390e9bf0059f/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:26.603098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.617612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 292/588] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:26.618713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.638986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.789697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f102-6fb1-42b5-9f90-ba114f4df02f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.800247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758583-43b6-418d-8b5b-b92fa998fa0e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.856687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.857908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.858411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.859094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.869173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.870748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.871257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.871755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.872505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.874029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.893270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:26.895443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:26.898718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:26.902548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:26.902548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:26.902548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:26.902548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:26.902548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 293/589] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:26.908053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.908367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:26.923998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:26.926120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:26.928972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:26.928972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:26.928972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:26.928972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:26.928972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 297/590] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:26.928972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.966957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.035410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.085430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872410a-1184-4874-93b2-6628416edf18 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.108344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a83db0-d1a3-42bb-9f50-5079b62a37df None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.147808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.149599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.150085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.151145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.167630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.169212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.170418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.171923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.173328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.173328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.174787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:27.175276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.177876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:27.199939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:27.202525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:27.206188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:27.206188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:27.206188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:27.206188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:27.206673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 298/591] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:27.206673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.206940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 294/592] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:27.216821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.413876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.442377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85b119-c4c8-4263-be77-3f85efd6a2c6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.454217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06a1d2-beae-4657-a899-23395a6cafe2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.489494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.524235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.525383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.525877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.526364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.527234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.527766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:27.528311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.529444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.530185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.530834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.531280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.546415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.550282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.558903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 299/593] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:27.560411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:27.583661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:27.585470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:27.587440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:27.598647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 295/594] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:27.600936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.809536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c5370-9169-4477-975b-3ee3d1ca842c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.895471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.895785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.896145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.896482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.898036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:27.898988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.902912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.961453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 300/595] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:27.963083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.993377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:28.005970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0793d0-f8ca-40b1-93c9-5bf9f7c21a32 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.129615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.130446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.130724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.130997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.152847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.154372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:28.189756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:28.191525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:28.193715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:28.200861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 296/596] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 601 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:28.201845 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:28.211479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefedcdd-a92d-4c76-b123-3a3691489e8d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.263256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.263941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.264226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.264508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.265751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:28.266682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.267765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.282381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:28.295009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 301/597] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:28.295901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:28.380841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:28.393357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64100bd9-a86d-4cde-a2e8-9068d8dbf515 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.432283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.433078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4b2-39ad-4dc3-b7fb-6c9c8229c0f8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.433246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.433382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.433645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.442551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.444072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6b4e2da5-4df5-4c3b-89d3-ff0d74915f7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:28.461070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:28.462135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 297/598] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:28.463764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2989d64-f78f-427c-a11b-bf0f5343c2d7 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.465367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2989d64-f78f-427c-a11b-bf0f5343c2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.465367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2989d64-f78f-427c-a11b-bf0f5343c2d7 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.465857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 298/599] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:28.466494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2989d64-f78f-427c-a11b-bf0f5343c2d7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:28.487929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.488744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.489224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.489694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.490633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:28.491188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.493055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.513104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 302/600] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:28.513104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:28.619768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:28.631429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2989d64-f78f-427c-a11b-bf0f5343c2d7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.655151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a4d8a-bb85-4586-a32a-78290f76294b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.679764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.680569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.680852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.681240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.694184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.694938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:28.695946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.711695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.712370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.712652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.712924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.713761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:28.714069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.715559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-50479e8c-3433-4c61-b991-ec576d690ca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:28.717221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:28.720727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 299/601] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.722998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.737751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 303/602] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:28.738547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:28.747814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:28.750640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:28.856760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f694b56-0385-4501-a73b-6dcffd36ed40 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:28.882906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 300/603] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:28.882906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:28.906885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:28.907925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.908230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.908487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/d529f6af558a44ef909937d5a960cb88/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.925652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=d529f6af558a44ef909937d5a960cb88, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:28.927292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:28.951099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 304/604] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:28 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/d529f6af558a44ef909937d5a960cb88/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:28.951099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.001582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.032355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.080366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b8592-eff7-491c-b377-cb7bc7646ff8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.131145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f614e2-f5c8-46fe-907c-572370fe2f50 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.159425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.160470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.161251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.161725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.163010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:29.164020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:29.167488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:29.192547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3ac45-2190-40d1-9bc2-bb6af0e5236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.193050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 301/605] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:29.193735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3ac45-2190-40d1-9bc2-bb6af0e5236a None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.194276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3ac45-2190-40d1-9bc2-bb6af0e5236a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.194582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3ac45-2190-40d1-9bc2-bb6af0e5236a None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/3df4115fcce342f2ae095ec1a2af4cca/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.196312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.214382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3ac45-2190-40d1-9bc2-bb6af0e5236a None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=3df4115fcce342f2ae095ec1a2af4cca, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:29.215920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3ac45-2190-40d1-9bc2-bb6af0e5236a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:29.236939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 305/606] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:28 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/3df4115fcce342f2ae095ec1a2af4cca/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:29.238395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe65ec-9b82-425f-9c7b-056aa695bc6e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.238736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe65ec-9b82-425f-9c7b-056aa695bc6e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.239102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe65ec-9b82-425f-9c7b-056aa695bc6e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.240912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 306/607] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:29.241846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe65ec-9b82-425f-9c7b-056aa695bc6e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.395427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.405469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbe197-40d1-4083-b1d1-950f6c387a19 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.437034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe65ec-9b82-425f-9c7b-056aa695bc6e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.465628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19fd8e-cb3f-4e44-ad57-62373473dafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.467364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19fd8e-cb3f-4e44-ad57-62373473dafd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.468044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19fd8e-cb3f-4e44-ad57-62373473dafd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.469094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19fd8e-cb3f-4e44-ad57-62373473dafd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.473792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.492307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19fd8e-cb3f-4e44-ad57-62373473dafd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ad19fd8e-cb3f-4e44-ad57-62373473dafd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:29.503000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:29.504051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:29.504937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:29.512085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.512085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.512085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.512085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.512496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 302/608] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:29.513582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.514068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.514567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.523727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:29.525603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:29.541938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:29.542904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:29.543917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:29.544352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:29.545992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:29.552642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 307/609] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:29.554010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.680900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 303/610] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:29.683726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.699223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d1b83-1a0c-4017-abdf-063850e23fa3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.765596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.767013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.768032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.769428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.771492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:29.772198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:29.776361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:29.795673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 308/611] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:29.798017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.813829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.875514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c4f5-df50-47ae-8401-b475ae945d76 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.904660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.915852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.916532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.916812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.917097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.917906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:29.918222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:29.920359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:29.934299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686e709-76ef-40ca-8a6c-620f620a621d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.940854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 304/612] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:29.940854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.981490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.982669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.983137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.983561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.997474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:30.007134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:30.008273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:30.009131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:30.011696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 309/613] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:30.013235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.064882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51be0a-f802-44f5-a7c0-f756e32cb49c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.097293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.114243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.115087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.115396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.115679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.130247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.132392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.143181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c510c5f-2746-4410-aa69-edf91e32ad73 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:30.155683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:30.157779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:30.158701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:30.158701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:30.158701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:30.158701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:30.162325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 305/614] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:30.164432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.193375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.194296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.195048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.195048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/7f5dceea71d1434796a105ea67773cd2/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.213761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=7f5dceea71d1434796a105ea67773cd2, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.215336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.219022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.242423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 310/615] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:30 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/7f5dceea71d1434796a105ea67773cd2/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:30.242423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.351079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d4f66-f110-42d6-9185-69d266dccf4a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.426587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.427447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.427804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.428156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.429029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:30.432739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.434330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.448702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 306/616] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:30.458077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.466856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe487-e46f-4cf2-aba1-990087d67ee5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.495128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.526935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.528061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.528626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.529054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.529952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:30.530815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.533973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.550902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 311/617] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:30.557779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.683253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.699746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ac7eb-29ea-4abe-8f7a-e410aba4cd35 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.822899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5da2f-1385-4377-862e-dc2561cf467c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.831612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.832995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.833504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.833999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.876595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.879350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.912721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.914081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.914575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.915085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:30.934408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:30.936392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:30.939001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:30.939001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:30.939001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:30.939001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:30.939505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 307/618] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 482 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:30.946098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.946098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.947485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:30.980743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:30.983166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:30.986142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 312/619] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:30.992727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.091190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.261399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89686fc-747e-492a-96a4-446a56d6ced6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.286718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c68a56-18c9-49c5-bc5d-e51a1f048861 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.336031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.337663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.338436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.339223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.340942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:31.341918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.346465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.372000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 308/620] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:31.382709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.407037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.407775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.408258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.408719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.410745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:31.410745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.413419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.416324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.482275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 313/621] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:31.488754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.611334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876ef6-8628-4da3-995b-b4dd3230886e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.687686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.690990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.696513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.697412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.717549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.722227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:31.753681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:31.755591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:31.757437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:31.762271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 309/622] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:31.773390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.827780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.848662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448c619-d55b-4a41-8d8d-2ff521f5e666 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.926483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.953234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.954209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.954541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.954697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.975927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.977653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.994456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404766-fd15-4b8b-a951-61336e45722c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:32.009434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:32.011290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:32.013038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:32.013038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:32.013038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:32.013038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:32.019434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 314/623] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 531 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:32.026022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.155424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.158046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.158507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.158953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.165392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:32.167965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.176288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd7e4c-902a-4dbd-b2bf-c50988328d2a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.226964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 310/624] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:32.226964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68757-d0cc-4b8d-b67b-5c42dead6ab5 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.226964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68757-d0cc-4b8d-b67b-5c42dead6ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.226964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68757-d0cc-4b8d-b67b-5c42dead6ab5 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.226964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 311/625] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:32.229777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc68757-d0cc-4b8d-b67b-5c42dead6ab5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.296093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a92c26-a6b7-4969-bf1f-96165dbc6eb8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.366525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.367830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.368455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.368941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.370417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:32.371032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.373857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b56d6-3167-41d4-a623-de51d5911c3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.400188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 315/626] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:32.402686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.403231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.403790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.434924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:32.437695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:32.441973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.463183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc68757-d0cc-4b8d-b67b-5c42dead6ab5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.528349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.541324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.542697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.543575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.544555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.585202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 316/627] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:32.587050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.698008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.699584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.808255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6ea1-edbb-473b-ab1e-498c2036cde5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.880483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.882950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.883643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.884318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.898607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.901714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.905214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.925653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 312/628] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:32.925653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:32.942474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:32.944357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:32.946111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:32.950375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 317/629] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:32.952271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.988474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.120230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d027c-fd65-4daf-9fce-25b1edeec043 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.168710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c1ce6-1345-48fd-9259-39e01d14a043 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.172438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.173773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.174228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.174687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.195907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:33.211378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:33.212602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:33.213485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:33.214124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 313/630] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:33.215180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.228371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.229746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.230260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.230732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.253412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.256044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-791df0e9-8c12-47be-a07f-2a4935dae44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:33.289648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:33.290609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:33.291543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:33.291543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:33.291543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:33.291543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:33.291764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 318/631] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:33.293874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad496a-e382-4ab4-97b8-5a2632eb4e44 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.294369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad496a-e382-4ab4-97b8-5a2632eb4e44 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.294829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad496a-e382-4ab4-97b8-5a2632eb4e44 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.297983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 319/632] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:33.300248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d4482-abbf-41b0-a208-9448421f5e2c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.300735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d4482-abbf-41b0-a208-9448421f5e2c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.301210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d4482-abbf-41b0-a208-9448421f5e2c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.307122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 320/633] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:33.307122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d4482-abbf-41b0-a208-9448421f5e2c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.346726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.362663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975501-b67c-44b7-8b66-77ba67a8c31f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.370474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.424271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.425386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.425864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.426358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.427935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:33.428488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.428807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d4482-abbf-41b0-a208-9448421f5e2c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.431364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.460507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 314/634] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.460507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.475145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.475840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.476130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.476411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.477179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:33.477490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.478925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd1d46-c77a-43a9-a295-68f5bce15ab6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.492662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 321/635] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.494011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34de4-ec03-45c5-9a39-26d37c6e8149 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.494300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34de4-ec03-45c5-9a39-26d37c6e8149 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.494606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34de4-ec03-45c5-9a39-26d37c6e8149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.513128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a6c34de4-ec03-45c5-9a39-26d37c6e8149 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:33.514359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a6c34de4-ec03-45c5-9a39-26d37c6e8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:33.599210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a55846-c69c-42c6-8031-d7d3b1db5ddd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.610086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 322/636] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:33.611210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.611509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.611790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.630016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:33.631160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:33.645265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.646241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.646651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.647486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.648520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:33.649105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.651201 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.667081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 315/637] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.667933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.679806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.690845 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.715520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 323/638] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:33.716400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.834594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c09e3-517c-4948-b683-fc9a9c5c75b7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.847719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f96a5-c44a-45f6-88b5-f6fbe4bfca2c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.888262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.889452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.890764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.891035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.899644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.900609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.901051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.901444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/08ed51a3aab34ac0bf5791fa3307a79d/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.905270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.908086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.920800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=08ed51a3aab34ac0bf5791fa3307a79d, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.922474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:33.928397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:33.930656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:33.932943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:33.932943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:33.932943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:33.932943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:33.932943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 324/639] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:33.932943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.942021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 316/640] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:33 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/08ed51a3aab34ac0bf5791fa3307a79d/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:33.942885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.989261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.044796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.121229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ea2-1733-4b89-8b5a-79fc9608e806 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.135005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839d9c8-9a07-4db0-ba24-c021971654d0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.181721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.182464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.182745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.183159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.191708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.193321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:34.209800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:34.211677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 317/641] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:34.213624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.219115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.219115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.219115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.219115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.240199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:34.257363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:34.259595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:34.261271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 325/642] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:34.262572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.396339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.416653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47667991-509c-4d86-a1d6-ff3199456319 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.419519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd806414-084f-4999-949b-441b57d74b3d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.448431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.469474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.470878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.471499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.472028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.479990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.480996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.481448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.481903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.483175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:34.484162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.486246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.492121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:34.504954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:34.506781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 318/643] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:34.509020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.513204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 326/644] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:34.514903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.680215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe340a-c769-4566-8738-a40446723c14 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.705131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83895de9-f436-4e0b-819b-35b3c39e1eec None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.724961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.725642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.725931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.726211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.727020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:34.727373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.728859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.743110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 319/645] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:34.743693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.771978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.773634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.774048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.774522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.776004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:34.776526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.779242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.795793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 327/646] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:34.797393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.811034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.819530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.935090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458b012-325a-4fe9-81c6-879c25db0e09 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.939377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069eeec-efca-4be8-a61c-a0b3bd9c6365 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.980325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.981041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.981324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.981609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.992454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.993990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:35.010137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:35.012064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:35.013981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:35.016726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 328/647] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:35.017022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.021194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.022549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.024241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.024241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.024705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:35.025209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.027623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.046917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 320/648] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.046917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.129559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.194947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634d3c8-b9b1-4d0e-ba85-34e19da5bc24 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.218699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6f1e2-9f41-44d7-a6a3-51f9785796ce None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.221170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.267895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.268859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.269311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.269677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.270905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=054c95ae7aa645b7ade6390e9bf0059f) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:35.271440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.274461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.286147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 329/649] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/role_assignments?user.id=054c95ae7aa645b7ade6390e9bf0059f&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.288208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.289561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.291389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.292379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.292887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.320099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.323910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:35.368386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:35.369952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:35.371514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:35.371514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:35.371514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:35.371514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:35.375348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 321/650] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:35.377030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.504176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1d4bf-fc3c-4353-89ca-13f6e3c79502 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.562755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.575046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.575949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.576883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.577398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.579357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:35.579889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.582416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.616389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbb8d6-a322-4229-bc72-008a0c68fd1d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.620438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 330/651] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.622445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.684229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.686477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.687922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.688703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.715507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.719308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:35.755585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:35.757552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:35.759625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:35.759625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:35.759625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:35.759625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:35.766771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 322/652] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:35.768820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.799285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.825756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894baca-c31c-4e71-a0a5-6eb1b08d24c3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.894657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.895486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.896367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.896564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.898127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:35.898836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.901403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.931499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 331/653] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.933253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.993084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b4856-5d76-4aea-beee-d86b039597eb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.037227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.067165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.068585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.069412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.070228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.138643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c76e4-17f6-4acd-aeb4-d2e4d09fbed1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.207975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.209606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.209606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.209958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.211289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:36.211829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.214339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.214719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.217929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.243096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 332/654] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:36.243096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.243505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.383308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a7c9-0ce0-46ab-a108-4b9c55b8e342 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.396464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.405363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 323/655] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:36.406543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.432806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.433628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.433910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.434193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.443311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.444860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d548f12c-43d2-4e90-b9cb-d987608b75c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:36.462010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:36.463039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:36.464039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:36.468075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 333/656] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:36.470049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb443323-619a-4b13-a200-212aee30481f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.470343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb443323-619a-4b13-a200-212aee30481f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.470626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb443323-619a-4b13-a200-212aee30481f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.476202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 334/657] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:36.476202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb443323-619a-4b13-a200-212aee30481f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.577077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83598688-ae99-4ef4-b74b-0b3853d0fac1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.588918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.599345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb443323-619a-4b13-a200-212aee30481f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.623366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.624660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.624953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.625363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.633354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.634830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:36.652086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:36.653920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 324/658] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:36.656487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.658731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.668677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.668970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.669262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.690540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.693628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-250a3a6e-779e-4a4d-9e07-ed66c38188bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:36.716075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:36.727022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 335/659] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.729139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.754501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:36.756472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:36.787097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.834171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa936c5-e4d6-4c7d-a983-2156869891ba None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.895545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 336/660] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:36.897349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.899873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.901045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.901548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.902038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.903828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:36.904267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.907563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.942603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 325/661] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:36.944182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.982312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.094886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd34f8-5497-4772-87cd-6d3ec37f33a8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.102922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f3cc2-ec4b-4c7c-8a8f-ff4d9bb0be9d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.139627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.140791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.141276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.141678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.142568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:37.143143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.144888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.161360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 326/662] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:37.167443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.172601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.173218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.173718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.174205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.175875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:37.176426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.178987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.217506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 337/663] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:37.217506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.233211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.320324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.353661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634b85-e3da-4b06-8055-4d982e28d6e6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.396092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaf53b-98c3-4cec-90d3-2ae7bf6d54ef None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.409458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.410415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.410782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.411193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.426771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:37.436512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:37.444870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 327/664] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:37.446751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.466811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.468697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.469817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.470253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.564376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.566253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.580419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.632570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37449d-5f12-4e3d-b950-3c403b61b6d9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.684664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.685839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.686298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.687288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.700499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.703379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:37.722097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:37.731590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 328/665] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:37.733692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.738811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.776857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 338/666] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:37.776857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.907942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b542b4-a632-46a1-a613-dadc3ff95d21 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.915539 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea511d56-534a-41ce-b91d-dc6365d9e63e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.922543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.958828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.959595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.960003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.960393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.972098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.972885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.973239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.973582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.974456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:37.974866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.976428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.987133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 339/667] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:37.988129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.067006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.068532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.072524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.109094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8e041-899d-4b70-9e98-e0953585a79c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.159194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.160349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.160858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.161366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.162915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:38.163525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.166355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.189883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 340/668] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.197271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 329/669] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:38.197271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.197612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.239833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.341594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bdba5-1ccb-4231-bc77-02690ce2ae0a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.341902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f86bf1-734d-49ed-9947-3d6cc13aeaaf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.346235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.388658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.390446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.390896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.391620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.407272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.409774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.410237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.410642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.410787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.413255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.428918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.431496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-17a8d985-d86b-45c8-859b-d5bb06f65cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:38.444196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:38.446009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:38.447991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:38.447991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:38.447991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:38.447991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:38.451257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 330/670] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:38.453736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de032978-f192-4216-adc6-9fbc6456b1a4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.454154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de032978-f192-4216-adc6-9fbc6456b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.454621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de032978-f192-4216-adc6-9fbc6456b1a4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.456229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 331/671] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:38.457493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de032978-f192-4216-adc6-9fbc6456b1a4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:38.461488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:38.463478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:38.471790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:38.471790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:38.471790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:38.471790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:38.471790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 341/672] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:38.471790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.582520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.593124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de032978-f192-4216-adc6-9fbc6456b1a4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.618126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65286a61-d25a-4c19-b680-d6835e7a1419 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.632034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.632838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.633124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.633395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.646355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.648123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.666397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.667569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.668085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:38.668230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:38.669246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:38.670384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:38.670384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:38.670384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:38.670384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:38.670656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.670729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 332/673] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:38.670875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=d529f6af558a44ef909937d5a960cb88) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:38.670973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.671078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.673408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b2c65-73f0-4374-be85-b96045987d55 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.684125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.707105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 342/674] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/role_assignments?user.id=d529f6af558a44ef909937d5a960cb88&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.707105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.707105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.707105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.732082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:38.735031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:38.819959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc1da0-b73d-4ead-8b3f-2adfa8d51526 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.839394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 343/675] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:38.840245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.893301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.894478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.894986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.895521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.896918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:38.897823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.900680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.921993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 333/676] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.923686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.954151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.998470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f12f15-8974-4fac-b393-d82cf88f0961 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.037547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.063621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.064367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.064856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.065339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.066726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:39.067300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.069929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.099301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 344/677] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:39.102445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.110113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fffe2-eedd-472e-95d8-fc2b6d14c37f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.184378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.185724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.186363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.188057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.189499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:39.190231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.193185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.220016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 334/678] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:39.221571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.235187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fe957-d45a-476d-bfd7-5f9f0cc2c477 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.282353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.283368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.283817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.284275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.288332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.301009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:39.311120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:39.313655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:39.315375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 345/679] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:39.317379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.404603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e278-0af1-4997-99a2-cad7882f4115 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.453732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.470522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.471938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.472404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.472903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.488045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.490892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.501556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c05f2-1fc9-47de-9d8e-15a6cec8a4dc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:39.518372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:39.520487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 335/680] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:39.523638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.563030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.564380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.564847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.565283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.578849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.581605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:39.609608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:39.611717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:39.613933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:39.620423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 346/681] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:39.620423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.692666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452d984-c593-4e9a-82de-fe4807796a3b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.737420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.742695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.743344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.743626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.743902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.753319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.754509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:39.770260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:39.772310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:39.774142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:39.776608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 336/682] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:39.778035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.828504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cb5a7-b89d-4beb-b1c3-52b699e8ab92 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.837926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.897519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.898713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.899258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.899778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.901312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:39.901849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.904668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.920659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 347/683] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:39.921473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.922292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a991fa-2590-473f-9773-fdd704c37ba8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.002912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.004386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.004880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.005398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.007116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:40.007679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.010358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.043991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 337/684] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.045302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.099147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae739b8-5c89-4138-a015-717c35cf77fd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.122087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.142213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.142905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.143232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.143489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.144323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:40.144632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.146117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.161730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 348/685] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.162342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.237887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a7701-1b01-480f-8688-8020e4581dab None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.247389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.282497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.283683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.284189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.284686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.286219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:40.286757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.289466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.312127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 338/686] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.313300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.313845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3d513-00e1-4916-aa73-f6e9a2b67dd2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.382811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.384247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.384730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.385237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.405038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.408119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:40.443623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:40.444638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:40.445555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:40.445555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:40.445555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:40.445555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:40.446267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 349/687] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:40.447385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.505049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.510599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b52a31-1c29-41c6-b9b9-ccc48613a963 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.556538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.557367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.557790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.558169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.599542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.644441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc2f3e-b5a7-439e-b476-3d97cd7078f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.663920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.665985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.718886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.720092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.720598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.721090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.722723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=3df4115fcce342f2ae095ec1a2af4cca) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:40.723653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.726209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.745700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 350/688] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/role_assignments?user.id=3df4115fcce342f2ae095ec1a2af4cca&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.745700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.848267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 339/689] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:40.849962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.956957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.965022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.975652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeb65c-2ef9-4050-9558-61dcdd8ccf69 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.008831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4537a-6429-4ee6-b844-10f0a4afeee4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.044614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.046237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.047015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.047769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.049613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=7f5dceea71d1434796a105ea67773cd2) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:41.050123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.053995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.064547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 351/690] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/role_assignments?user.id=7f5dceea71d1434796a105ea67773cd2&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:41.065967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.073557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.075016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.075736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.076399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.078631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:41.079465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.082330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.105373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 340/691] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:41.115740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.295607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135e8a8-56f8-4770-9d4d-e484e688d9b1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.318415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25883b03-de42-4da5-92b7-8ca2a2e21b23 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.345112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.346069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.346358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.346638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.360958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.362469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.373869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f39cc3ee-64d5-42ef-b745-5721604da8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:41.383988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:41.385046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:41.385942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:41.385942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:41.385942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:41.385942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:41.385942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 352/692] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:41.385942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65329add-1a0e-4362-a9ae-d7a211d09ca7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.386288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65329add-1a0e-4362-a9ae-d7a211d09ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.386365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65329add-1a0e-4362-a9ae-d7a211d09ca7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.388708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 353/693] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:41.397181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.397525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.397753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.402462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.403869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.404364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.404862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.417493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:41.418805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:41.419567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.422474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.423529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:41.451277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:41.453228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:41.455042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:41.459773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 341/694] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:41.461663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.509009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 354/695] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:41.510085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.651690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afaeab-8260-4c27-8cb7-2a762937b1fb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.657871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb60fb-07cc-45cf-8a43-d06a911dd0cf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.707434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.708590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.709142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.709624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.710734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:41.711298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.712999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.726615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.738195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 355/696] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:41.739431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.743307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.745959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.746469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.746970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.748666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:41.749783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.753623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.784420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 342/697] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:41.785835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.872917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209f7e-9cf2-434a-a0f3-4813db8d84d5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.893141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.916299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.917064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.917351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.917629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.918321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.919827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:41.936815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:41.938776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:41.940504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:41.943207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 356/698] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:41.944258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.019468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0a383-cfba-4461-9e68-a7488cdafad2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.020311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.071382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e65e35-917c-4c51-9a5a-649e83c4c3ff None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.092728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.094426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.095253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.095957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.114781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.115654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.115942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.116224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.125382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.126940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:42.143488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:42.145505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:42.147809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:42.149922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 357/699] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:42.150941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.247282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.251994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.268877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922e8fc-daa8-4b1a-8322-686a290b7bf8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.295717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.306698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.310602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.311901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.313059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.313348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.314086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:42.314395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.315841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.326450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 358/700] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.327447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.369444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 343/701] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:42.370562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.453616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275fb8f-772a-45bb-ab71-f35899d343a5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.488019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec112-0cf8-4226-96e2-ad828e6df31e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.495070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.495756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.496043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.496320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.497378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:42.497560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.499104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.519129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 359/702] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.520130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.528544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.529493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.529924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.530868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.531997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:42.532479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.534025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.540835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 344/703] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.541905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.576446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.621411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.705218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01343-3f10-48ab-b3a0-cbbdd3fdab19 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.747370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.748107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.748331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.748607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.750091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=08ed51a3aab34ac0bf5791fa3307a79d) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:42.750406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.752122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.759251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 345/704] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/role_assignments?user.id=08ed51a3aab34ac0bf5791fa3307a79d&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.816817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc7730-33d5-4600-88aa-f3150f949cc7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.889957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.891516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.892022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.892503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.913304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.915918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:42.954131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:42.955161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:42.956064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:42.956064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:42.956064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:42.956064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:42.960992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 360/705] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:42.966907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.068496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.103374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a789059-b17d-49c7-ae05-babcd9ced1b3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.143407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.144090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.144373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.144648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.145385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:43.145692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:43.147358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:43.161543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 346/706] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:43.167112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.310921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.320605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a873aa-770c-4b84-9ab0-b2de35d3a640 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.397313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.398280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.398568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.398933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.416401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:43.419164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:43.453428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:43.454852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:43.455797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:43.456569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 361/707] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:43.465691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.703538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b174e5a-0156-4278-8faf-38d85410bd3a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.765543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.766478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.766763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.767039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.767881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:43.768196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:43.769689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d49aa-baf1-4574-8836-08a860ce4e47 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:43.771626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.790721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 347/708] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:43.798789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.866810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe331c-cb18-4078-a4eb-003269c1d43f None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.867171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe331c-cb18-4078-a4eb-003269c1d43f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.867399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe331c-cb18-4078-a4eb-003269c1d43f None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.869103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 348/709] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:43.880585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.880917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.881144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.914893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:43.916068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:43.987671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c4643-1b22-4bf1-9ce7-dd79e3e57bd4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.087017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.087722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.088008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.088289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.089141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=054c95ae7aa645b7ade6390e9bf0059f) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:44.089453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.090941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.098428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 349/710] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:44.103345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 362/711] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/role_assignments?user.id=054c95ae7aa645b7ade6390e9bf0059f&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:44.122731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.293188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0388b-a993-4e76-aca0-b0dc1c3889f5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.352962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.380311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.381626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.382083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.382534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] PATH_INFO: `/v3/services/network` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.383713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.386065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:44.414118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:44.415256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:44.416150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:44.417006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 350/712] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:44.424902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.462899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.579204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76b75b-326f-44ec-88c3-a43060b9746c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.615599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.616287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.616581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.616865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.617589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:44.617902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.619337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f0bed-be7d-4e87-ac53-935889c55789 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.625112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 363/713] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:44.632705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.641133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ddd8-b77f-4987-88d1-cd7efedf701d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.641449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ddd8-b77f-4987-88d1-cd7efedf701d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.641705 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ddd8-b77f-4987-88d1-cd7efedf701d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.643398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 364/714] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:44.655724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.656058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.656283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.683383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:44.684618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:44.808805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 365/715] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 22:19:44.856733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.900382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.901917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae86fd-f55b-45dd-987f-3a4b021e6245 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.968290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.969274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.969724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.970152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.971021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] RBAC: Adding query filter params (type=network) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:44.971508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.973188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.986907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 351/716] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:45.066236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1eaa3-4483-42c6-b62a-4bf1a0cc3beb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.134495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.135840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.136352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.136858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.160287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.163397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:45.198751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:45.199849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:45.200825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:45.200825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:45.200825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:45.200825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:45.201545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 366/717] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:45.211185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:45.327689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:45.427368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f16e2d-31f1-44db-b818-1d973e44ef2f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.511386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.512566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.513046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.513532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.515485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:45.516025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.518682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f6a39-8a50-4675-a89b-26b69308ff88 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.555180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 352/718] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:45.564217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:45.685609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:45.749124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1679-2a0d-4dac-9efe-2da31550c607 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.815372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6fdee-edfc-42fd-a594-f98471b51be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.816538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6fdee-edfc-42fd-a594-f98471b51be0 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.817010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6fdee-edfc-42fd-a594-f98471b51be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.817499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6fdee-edfc-42fd-a594-f98471b51be0 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.818813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6fdee-edfc-42fd-a594-f98471b51be0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.821532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6fdee-edfc-42fd-a594-f98471b51be0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.855506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 367/719] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:45 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:46.082752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ced75-a2a2-4450-89ed-0b8902a032ab None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:46.083157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ced75-a2a2-4450-89ed-0b8902a032ab None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:46.083588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ced75-a2a2-4450-89ed-0b8902a032ab None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:46.086523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 353/720] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:46.093600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:46.094152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:46.094610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:46.133590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:46.135747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:46.260501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 368/721] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:46.288043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ced75-a2a2-4450-89ed-0b8902a032ab None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:46.498880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ced75-a2a2-4450-89ed-0b8902a032ab None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:46.525134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:46.565531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f689ae7-3698-484c-9bc9-c520d74355eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:46.566894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f689ae7-3698-484c-9bc9-c520d74355eb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:46.567430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f689ae7-3698-484c-9bc9-c520d74355eb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:46.567925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f689ae7-3698-484c-9bc9-c520d74355eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:46.587967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f689ae7-3698-484c-9bc9-c520d74355eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-4f689ae7-3698-484c-9bc9-c520d74355eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:46.604253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:46.605281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:46.606141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:46.607395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 354/722] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:46.616227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:46.677393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009831-c5b7-4d8b-a95e-a9d5ff609555 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:46.677894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009831-c5b7-4d8b-a95e-a9d5ff609555 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:46.678329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009831-c5b7-4d8b-a95e-a9d5ff609555 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:46.681511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 355/723] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:46.694774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:46.699179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:46.699179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:46.725397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:46.729920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:46.732372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:46.844698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dd751-9eed-4341-b53e-5d19dc97a7d8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:46.882732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 356/724] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:46.910718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:46.911445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:46.911729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:46.911998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:46.912770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:46.913081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:46.914661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:46.921983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:46.940126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 369/725] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:46.945518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:47.127777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a597-c46d-4cfd-af6f-cff30f3ca119 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.135673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48840a-06bd-40d3-baa9-443a04a6ccb8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.137960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:47.149281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:47.179400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:47.180209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.180527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.180745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.183245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64910b3d-c928-4169-b4bd-04026e308d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:47.184051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64910b3d-c928-4169-b4bd-04026e308d37 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.184407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64910b3d-c928-4169-b4bd-04026e308d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.184755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64910b3d-c928-4169-b4bd-04026e308d37 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.185564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64910b3d-c928-4169-b4bd-04026e308d37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:47.187132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64910b3d-c928-4169-b4bd-04026e308d37 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:47.195336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:47.196828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:47.214366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 357/726] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:19:46 2022] POST /identity/v3/services => generated 251 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:47.216458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6350b-0d3c-4d41-949b-24b9977640bb None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.216962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6350b-0d3c-4d41-949b-24b9977640bb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.218207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6350b-0d3c-4d41-949b-24b9977640bb None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:47.222089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:47.223050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:47.224005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:47.224005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:47.224005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:47.224005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:47.224005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 370/727] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:47.226403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 358/728] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:47.227443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:47.239816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.240447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.240930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.273095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:47.274918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:47.370560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3fda-67a9-46c7-89a0-7956b79e7569 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.427546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:47.428146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.428432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.428711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.429444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:47.429751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:47.431520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:47.448402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 371/729] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:47.451471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 359/730] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:47.455302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:47.479865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:47.486624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:47.597372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14208eca-53bc-4f51-9b4d-deccdab3be59 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.643006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:47.650260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:47.651923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.652298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.652837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.666746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42680a78-e39d-4b41-a4a5-e280dab2642f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.674030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:47.675730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:47.694288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:47.695612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:47.696707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:47.703588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 372/731] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:47.709509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:47.721792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:47.722901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.723957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.724455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.741620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:47.755099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:47.756311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:47.757333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:47.760765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 360/732] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:47.769389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:47.834793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:47.854972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77863f8-5ed8-41a8-9300-54b21f25748c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.903640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:47.905293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:47.907297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:47.907738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:47.908775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:47.911316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:47.913015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:47.956060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 373/733] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jul 21 22:19:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:47.958435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3e6d-c5e2-4d01-b468-f32e50a1cc49 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:47.961871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.038239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.040297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.042046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.042482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.047146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:48.049008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.050719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.072237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 361/734] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:48.078888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.110048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:48.166252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefffd2-5983-44b8-aed0-625db49b852c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:48.211104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.212084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.212567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.213022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.242582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:48.307523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d7734-e4bb-4680-ba56-147ac67e22db None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:48.357657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.360264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.360741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.361217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.375674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.377512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:48.398884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:48.400086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:48.401267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:48.401267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:48.401267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:48.401267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:48.401493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.403213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.403714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 362/735] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:48.409692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.603754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 374/736] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:48.607426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:48.612445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.664637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58314a63-618f-42ae-b661-1de83508d097 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:48.719883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:48.744962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.746780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.747776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.749204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.750991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:48.751965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.754728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f3bf0-b4c3-49c6-9b0f-ba6ebdd6fbeb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.782873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 363/737] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:48.787170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2110b-aaf3-4b68-94bb-3c14aa3d2bb4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:48.790123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7cc9c-2425-4115-bfa3-773c37f6037b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.790864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7cc9c-2425-4115-bfa3-773c37f6037b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.794662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7cc9c-2425-4115-bfa3-773c37f6037b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.798270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 364/738] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:48.800261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7cc9c-2425-4115-bfa3-773c37f6037b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.827909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.828915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.829382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.829681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.841473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.843245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-dc0d3b03-1db0-45a4-9a71-d1bbf59866da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:48.862754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:48.864844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:48.866755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:48.866755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:48.866755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:48.866755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:48.870514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 375/739] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:48.872138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.872536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.872921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.891720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:48.893101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:48.971352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 376/740] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:48.972476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.015907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.027987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7cc9c-2425-4115-bfa3-773c37f6037b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.086294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.103298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ce4c-fdc4-41ea-8f17-8ddc8f0e4008 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.106943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.108720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.109470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.110217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.127293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.130668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.142236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.143090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.143516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.143910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.144756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:49.145167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.146975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.163343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 377/741] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:49.164414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:49.164864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:49.166590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:49.168453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:49.173161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 365/742] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:49.181009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.335785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39738581-a084-4dd2-b49a-add5e39e9974 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.339885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.379725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.380498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.380781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.381109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.383547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e1c27-c7cc-416b-9c43-51a78bccbd8d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.391829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.393343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:49.409704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:49.411557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 366/743] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:49.413245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.446784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.447592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.447878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.448160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.448986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.450514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.475259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 378/744] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:19:49 2022] POST /identity/v3/services => generated 240 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:49.477557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.572933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.731340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75aea1e-df81-4209-bfe7-68aee9004899 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.796139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.796848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.797261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.797547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.798777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:49.799119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.800622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.809210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f38b2c-7bcd-4923-b8d7-1e541f485b85 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.825551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 367/745] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:49.833080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.891636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.893220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.893986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.894731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.896316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:49.897086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.899709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cdfc7-6288-4979-aaa8-32645dea423e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.922656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 379/746] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:49.924909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1603d3-530a-4160-bd20-55c416f943d3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.925380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1603d3-530a-4160-bd20-55c416f943d3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.925844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1603d3-530a-4160-bd20-55c416f943d3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.928782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 380/747] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:49.931362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1603d3-530a-4160-bd20-55c416f943d3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.939202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.019543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.026580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cf8d9-f877-4af3-94b4-06104c19726f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.089079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.090512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.091125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.091671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.111818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.114335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.129898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1603d3-530a-4160-bd20-55c416f943d3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-88f34e40-a52e-4f90-8496-e66c522fb1b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:50.147170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:50.148179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:50.149078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:50.149078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:50.149078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:50.149078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:50.149078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 368/748] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:50.151516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a790-c0ec-4ffa-a28e-3e82096c3975 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.152160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a790-c0ec-4ffa-a28e-3e82096c3975 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.152736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a790-c0ec-4ffa-a28e-3e82096c3975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.181132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-13b7a790-c0ec-4ffa-a28e-3e82096c3975 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:50.183165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-13b7a790-c0ec-4ffa-a28e-3e82096c3975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:50.202143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.203432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.203980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.204533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.206154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=08ed51a3aab34ac0bf5791fa3307a79d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:50.206776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.209520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.220811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 381/749] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/role_assignments?user.id=08ed51a3aab34ac0bf5791fa3307a79d&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:50.223232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.319644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 369/750] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:50.322711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563592e-7c61-4e95-b87a-18b842b5502a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.323258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563592e-7c61-4e95-b87a-18b842b5502a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.323894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563592e-7c61-4e95-b87a-18b842b5502a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.335689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 370/751] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:50 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:50.336937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.343874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.345249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.345958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.384492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:50.387330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:50.390177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2495ed0-03b0-417c-b11d-1f35d49191a3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.464232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.465289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.465676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.466057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.466939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:50.468421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.470009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.490308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 382/752] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:50.491272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.509033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 371/753] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:50.512351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.541151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.641549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72870655-2dc3-46e0-8be8-521acd4da696 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.711503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.714378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ea66f-81c9-404e-aaed-64d640156ec0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.722163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.723569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.724678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.725578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.748983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:50.766939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:50.768819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:50.770455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:50.775906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 383/754] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:50.777795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.778092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.778913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.779248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.780074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.788152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.789673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:50.805667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:50.807545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 372/755] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:50.809459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.889738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.926682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4b4d5-1028-4a19-b14e-be34bcdd2dab None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.964798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.965419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.965707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.966003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.966772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:50.967214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.968679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.978529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 373/756] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:50.979480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.990384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25b13-7c7b-489d-9a5f-0af7a3c69b0b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.062202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.064016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.064823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.065512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.067146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.069798 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:51.095836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:51.097714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:51.099954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:51.101476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba9bea-e894-4268-b6ac-a22ecbfdef96 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.104345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 384/757] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:51.106435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.141197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.141915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.142205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.142478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.143341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:51.143665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.145187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.147447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.161996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 374/758] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:51.163120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.220781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.287695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7c78f-3c30-4f36-a6b4-e881ee41a43c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.325800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac83f93-5bbe-43f1-87f1-511864140575 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.328593 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.329288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.329573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.329853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.330585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:51.330896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.332411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.338214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 375/759] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:51.339117 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.402298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.404298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.405219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.406054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.411357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.429184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.432630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.462471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11239b-4ebf-487b-afcb-d6498532221b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:51.470488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:51.472376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:51.474235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:51.474235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:51.474235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:51.474235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:51.478325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 385/760] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:51.480396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.504812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.505722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.506010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.506298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/054c95ae7aa645b7ade6390e9bf0059f/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.508373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=054c95ae7aa645b7ade6390e9bf0059f, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.509846 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.538843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 376/761] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:51 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/054c95ae7aa645b7ade6390e9bf0059f/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:51.542891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.705671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa80da-7d2d-4031-a91b-988730c10d96 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.734539 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.743219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a460fe2-bf9c-42fa-84bc-36de1f1e5430 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.753025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.757654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.758779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.759346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.759869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.760897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:51.761480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.763233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.780497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 377/762] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:51.786202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.872484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.873752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.873878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.874114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.874858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:51.875202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.876634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.901190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 386/763] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:51.907532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.959997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea08f0c-3f7c-4253-8e57-081d3f86ab1e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.030136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.031413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.031895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.032742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.051115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.054218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:52.087244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:52.088295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:19:52.089270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:52.090332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 378/764] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:52.096827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.097933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d9dcc-24dc-4ee7-9bc0-7e81e3bf963b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.153376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.155212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.155732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.156365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.157399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] RBAC: Adding query filter params (interface=public, service_id=69c722c01a4e49bd8d8ae279a0183ed6, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:52.157963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.159693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.175260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 387/765] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/endpoints?service_id=69c722c01a4e49bd8d8ae279a0183ed6&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:52.180699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.194791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.235835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.301702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575c79d-83bb-48b8-832c-1ee2d9233ffb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.326152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d999c8-29ce-40b3-8084-ff64aa370524 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.341258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.342938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.343402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.343905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.345057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:52.345585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.347297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893716d5-12eb-4d7c-9fae-9c725023eb08 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.368452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490c0e5-8f38-4df6-a304-18a2b8090d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.368978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 379/766] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:52.369400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490c0e5-8f38-4df6-a304-18a2b8090d75 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.369779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490c0e5-8f38-4df6-a304-18a2b8090d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.370155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490c0e5-8f38-4df6-a304-18a2b8090d75 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.370949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490c0e5-8f38-4df6-a304-18a2b8090d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.371156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580b4b-f731-4641-afa1-63915cff38f0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.371601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580b4b-f731-4641-afa1-63915cff38f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.372033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580b4b-f731-4641-afa1-63915cff38f0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.372782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490c0e5-8f38-4df6-a304-18a2b8090d75 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.377940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 380/767] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:52.379174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580b4b-f731-4641-afa1-63915cff38f0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.379935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 388/768] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:52.381393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.381785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.382167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.402011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:52.403630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:52.488573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 389/769] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:52.513253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.514432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.525332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580b4b-f731-4641-afa1-63915cff38f0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.549435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.583000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.583618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.583904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.584186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.704565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b8e38-8bdc-45c5-ab1d-f2d2dc0584c5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.710104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.711977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.745166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.745987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.746270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.746555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.747261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.749225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:52.762380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:52.765019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:52.766726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 390/770] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/services/network => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:52.771507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.882964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.894738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcbb05-4b0a-4122-9efc-5c59b405204a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.904113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.917907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 381/771] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:19:52.924401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.938901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.939898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.940336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.940762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.941633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:52.942094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:52.943708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:52.950015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 391/772] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:52.957090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.124020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344817-c82d-4b35-836b-ff2ed30aa855 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.135415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd91e86-116a-49a8-8d4b-ab44caf4d79e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.196381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.197464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.197968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.198454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.199411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] RBAC: Adding query filter params (type=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:53.199958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.202739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.203249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.204613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.205246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.205563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.218528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 392/773] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:53.225194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.226307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.230531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:53.270716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:53.271989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:53.272917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:53.272917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:53.272917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:53.272917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:53.273638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 382/774] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:53.279671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:53.281980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.416882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b004126-fb6a-4178-8cf8-63f7a1f37b3d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.418818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:53.477271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.478372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.478978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.479504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.480512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] RBAC: Adding query filter params (interface=public, service_id=ab45157a993e4220a644580f48abc3ac, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:53.481048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.483228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d6477-03a9-4e52-bf81-688dceb5361c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.494673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 393/775] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/endpoints?service_id=ab45157a993e4220a644580f48abc3ac&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:53.496748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba72e0-a4c4-4587-b275-c87eca62351e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.497319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba72e0-a4c4-4587-b275-c87eca62351e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.497829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba72e0-a4c4-4587-b275-c87eca62351e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.504692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 394/776] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:53.506021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba72e0-a4c4-4587-b275-c87eca62351e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.515627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f6d7d-bf21-4583-b1c1-6da9ded15195 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.562809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.563558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.563843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.564118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.564873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:53.565190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.566637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9bf3b-b0bb-4957-b0a3-3dd461aee122 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.582328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 383/777] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:53.583836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.584129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.584411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.605565 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:53.606805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:53.691222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 384/778] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:53.691222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.757310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:53.787327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:53.823726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba72e0-a4c4-4587-b275-c87eca62351e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.830413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d560f-b0f7-4ac2-9005-56afcff49817 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.868893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.869700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.869985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.870259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.878268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.879969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:53.895867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:53.897897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:53.899723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:53.901569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.902166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 385/779] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:53.903183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.904244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.905229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.906001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.907734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.910861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.927388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 395/780] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/services => generated 772 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:53.929695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.020451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3cc4d-efb8-4b23-95a1-a923b38f5837 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.032713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.064545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.065368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.065682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.065936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.082183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:54.091555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:54.094025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:54.095750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:54.103423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 386/781] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:54.105005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.190549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398300f-3f57-4a63-bf57-7e92083dec66 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.234383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.235088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.235379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.235662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.236810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:54.237130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.238609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.255195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 396/782] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:54.262942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.280284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.384925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755823a-062b-4e13-b560-90b59f87d497 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.395432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f6e5b-eeab-4919-811a-28d1de0a04b1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.442068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.442756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.443038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.443386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.444338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=48141af292e144caa3ea38613964403e) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:54.444658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.446180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bc289-2e28-44e5-bbe7-1fb616312e3d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.454346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 397/783] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/role_assignments?user.id=48141af292e144caa3ea38613964403e&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:54.455780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36396b-d3ad-470f-bddb-94c0177ad51c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.456168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36396b-d3ad-470f-bddb-94c0177ad51c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.456451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36396b-d3ad-470f-bddb-94c0177ad51c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.458086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 398/784] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:54.464047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.464476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.464646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.472476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.473718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.474210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.474706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.476273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:54.476830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.479615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.485021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:54.486308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:54.520074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 387/785] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:54.526762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.568205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.574912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 399/786] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:19:54.597935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.602719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.730783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51454f0a-9882-40f4-8d34-1f598b694ba6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.774603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.775451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.775736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.776011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.789448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:54.798510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:54.799546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:54.801483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 400/787] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:54.806055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.819985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf3612-2051-4148-a3b2-b13f79a9f9ff None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.879797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.924928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.926150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.926649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.927169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.932032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec9116-213e-4aa9-a3cf-a839b5679432 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.954304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.956213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.976827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.977524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.977846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.978127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.978895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:54.979333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.980740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:54.987114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:54.988852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:54.990608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:54.990608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:54.990608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:54.990608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:54.992124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 401/788] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:54.995235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 388/789] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:55.000813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.001184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.136247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.149163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c8f7a-3406-48b2-aed4-4f745a2dacbf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.187647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.191610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.192791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.193057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.193315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.194799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06b4a0-5a98-4671-beea-9cc2d9a30205 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.206114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.207648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:55.230994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.232083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.233793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:55.233793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:55.233793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:55.233793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:55.235745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 402/790] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.241489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.287332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.288337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.288771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.289196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.290116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:55.291507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.293128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.323003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 389/791] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:55.324235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.462251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cf1e4-829b-4aa7-8f90-80e70a54b990 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.538586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.539554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.539988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.540411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.541321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:55.542609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.544213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.547612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31c618-61c9-41a2-b0a5-dd71d3ac1ea4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.565513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 403/792] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:55.571472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.616691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.628856 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.630131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.630424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.630706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.639673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.653810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.656377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:55.722716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.724560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:19:55.726296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:55.743213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 390/793] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.758567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.834808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f988652-836c-477c-abfa-4964d77d1327 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.904754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.906622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.907467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.907942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.927482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.929449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-db8d43e2-a3f9-4db3-b568-8b3b8008477e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:55.953132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.954450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:55.956233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:55.958628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 404/794] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.961397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf4f1b-ab2a-444e-afde-8014eae40c7f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.962136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf4f1b-ab2a-444e-afde-8014eae40c7f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.963099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf4f1b-ab2a-444e-afde-8014eae40c7f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.964743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 405/795] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:55.965724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf4f1b-ab2a-444e-afde-8014eae40c7f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.992055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738765ab-24fc-4179-9781-4ea72454439f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.056191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.057245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.057693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.058132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.059599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.060089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.062834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff4507-9054-4d47-aba8-8f2f8d607349 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.103296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 391/796] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:56.105807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.106282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.106754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.131859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.134722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:56.136818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:56.143454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf4f1b-ab2a-444e-afde-8014eae40c7f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.181916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.183091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.183378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.183801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.184796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.185109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.186900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.211122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 406/797] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:56.211998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.230763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.275493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 392/798] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:56.277247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.333450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d0daf-dcd4-4b82-b0c6-9312e7cd2ac8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.380483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.381293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.381579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.381854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.395246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.397278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:56.421736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.422882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.423870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:56.423870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:56.423870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.423870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:56.423870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 407/799] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.427158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.434477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.516615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c060f0-a083-4ec5-a886-dbba8f6fecb8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.551119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f875fc-3861-4dd2-8c60-4e2de20abf5e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.583207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c21fa-4268-4ba4-91d8-1f26f8974766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.584492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c21fa-4268-4ba4-91d8-1f26f8974766 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.585128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c21fa-4268-4ba4-91d8-1f26f8974766 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.585631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c21fa-4268-4ba4-91d8-1f26f8974766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.594899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.595838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.596199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.596588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.597325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.598787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.614448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.615920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:56.617496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 408/800] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.619085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.694714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.726464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.728807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c21fa-4268-4ba4-91d8-1f26f8974766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.730941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c21fa-4268-4ba4-91d8-1f26f8974766 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.773435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ea65-d0cb-480f-972d-162f86833201 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.818832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.819465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.819927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.820112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.820729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.821039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.822435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.843458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 409/801] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:56.844821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.924271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 393/802] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:19:56.924271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08327c6-5036-41c0-a2e5-07607e892a38 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.924271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08327c6-5036-41c0-a2e5-07607e892a38 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.924271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08327c6-5036-41c0-a2e5-07607e892a38 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.924271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 394/803] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:56.924271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08327c6-5036-41c0-a2e5-07607e892a38 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.985541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3e62c-6378-4e16-b8c0-465cedc9e349 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.015561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.026711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.031486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.032495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.033289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.033717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.034598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.035043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.037315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.050924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 410/804] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.050924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.062903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08327c6-5036-41c0-a2e5-07607e892a38 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.111923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.112721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.113006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.113275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.124338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.127156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-27fdd97b-fbd1-4e89-8f0a-28b2af11da1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:57.152035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:57.155021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:57.156875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 395/805] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:57.157870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.157870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.158983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.174796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae04bb-2c52-491a-bfdc-9b381117011f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.185096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:57.187762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:57.217350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.218328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.218756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.219036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.233964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.236814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:57.267807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:57.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:57.270960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:57.270960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:57.270960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:57.270960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:57.270960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 411/806] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:57.272076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.289008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 396/807] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:57.289694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.351600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.373582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.408189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26c15-a8d4-47ab-8945-d0425acf6953 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.420404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f4b59-fc74-4162-b3a9-eee7b4b8b821 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.446228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.447241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.447241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.447474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.448548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.448662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.450056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.460791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.461463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.461743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.462016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.462735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.463043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.464462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.464601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 397/808] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.465326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.470014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 412/809] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.471085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.609008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c3f5f-31f1-4aaf-a3ca-25af3b1c1d05 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.626451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f392f8-9f12-4c53-844b-de49677b5140 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.638126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.652858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.664501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.664745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.665130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.665512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.666663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.667130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.668743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276216e-c2b6-40e7-94c9-21745b5330a0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.676073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.676897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.677186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.677464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.678146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.679636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.687901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 398/810] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.690035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869e80-d282-4b42-9011-bf3a62d192ba None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.690550 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869e80-d282-4b42-9011-bf3a62d192ba None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.691224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869e80-d282-4b42-9011-bf3a62d192ba None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:57.693123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:57.694495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:19:57.696377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:57.697939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 399/811] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:57.698082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 413/812] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:57.698082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.699229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869e80-d282-4b42-9011-bf3a62d192ba None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.830135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412297bf-be29-4701-ba3c-51f478cfd4ea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.842817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869e80-d282-4b42-9011-bf3a62d192ba None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.871866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.872838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.873250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.874176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/08ed51a3aab34ac0bf5791fa3307a79d/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.875864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=08ed51a3aab34ac0bf5791fa3307a79d, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.877528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.895436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 414/813] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:57 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/08ed51a3aab34ac0bf5791fa3307a79d/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:57.896759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.943829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.027232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435df57-097d-4da7-9b4c-d946886a8ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.028639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435df57-097d-4da7-9b4c-d946886a8ba4 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.029761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435df57-097d-4da7-9b4c-d946886a8ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.030415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435df57-097d-4da7-9b4c-d946886a8ba4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.032064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435df57-097d-4da7-9b4c-d946886a8ba4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.034798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435df57-097d-4da7-9b4c-d946886a8ba4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.064350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8def0-ae01-4b77-9edd-67a333abbbf4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.078841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 400/814] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:19:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:58.081276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.082004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.082499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.119266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:58.121455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:58.133813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.134587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.134874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.135188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.144503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.146053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:58.165305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:58.167431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 415/815] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:58.170532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.218457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.279913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 401/816] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:19:58.281526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.300782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.318237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062c9fa-154e-412a-badb-31056ef5a86a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.376322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.377443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.378819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.378819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.380698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.381073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.384760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.398116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 416/817] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.399457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.492947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df148e69-45aa-4b29-98a5-657c509a5ee0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.551976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.553152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.554291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.555160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.557063 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.558320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.562579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.581131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82458-c785-4f3a-94ca-aed9932d2156 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.594592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 402/818] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.595600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.644693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.645412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.645700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.646101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.658415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.668153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:58.677156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:58.678188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:58.679023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:19:58.681540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 417/819] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:58.682690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.723338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.739794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503938fa-cb73-4785-8ed5-0a9a24d23355 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.778429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.779322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.779726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.780009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.780744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] RBAC: Adding query filter params (type=network) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.781051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.782454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.788377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 403/820] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.789244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.846068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3d83a-717e-4d2b-908a-b4c2cdcd8474 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.884804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.885513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.885812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.886081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.886936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=054c95ae7aa645b7ade6390e9bf0059f) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.887312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.889204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.896584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 418/821] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/role_assignments?user.id=054c95ae7aa645b7ade6390e9bf0059f&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.897020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.897355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.900898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bc9b6-4cbd-412a-bd2d-bcf363f0b498 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.942726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.943864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.944304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.944787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.945746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.946239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.947885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.959229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 404/822] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.972320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.015599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.039986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210a1b9-dfa5-492e-8498-d89625a7b4d4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.079292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.080009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.080298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.081110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.081355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.082803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.107409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 419/823] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:19:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:59.259418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04ace1-39a7-4a93-bad3-7ef737184c52 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.341141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.343875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.344309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.344820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.366083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.368955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:59.404694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:59.405690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:59.406584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:59.406584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:59.406584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:59.406584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:59.411550 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 405/824] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:59.415856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.539468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e94e-dbc7-4c61-b044-750ae4af2f36 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.574168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.585566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.586259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.586594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.586877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.587655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:59.587969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.589376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.605314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 420/825] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:59.614736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.643492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.848712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982313a-dbd5-4d13-af21-29cd3633de35 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.912779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.914529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.915276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.915974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.931857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.934953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:59.965640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:59.968334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:19:59.970257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 406/826] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:59.983228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.186121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e63ad-6f2f-4167-8b89-a4decc734d7b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.257422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.258659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.259454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.259970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.261638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:00.262196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.265722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea8a9f-b5dc-4cf2-a7a9-e6112cb780fa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.303406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 421/827] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:00.313200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.446945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:00.511647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c451639-cc1a-4c27-a674-044a3fdd2442 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.551221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d2986-1129-40d5-a700-d16ae3f8848d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.552981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d2986-1129-40d5-a700-d16ae3f8848d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.553775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d2986-1129-40d5-a700-d16ae3f8848d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.556887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 422/828] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:00.563938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.564343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.564496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.581862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.583910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.584361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:00.584989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.585588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:00.586072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.606652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.609843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:00.646126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:00.648144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:00.649905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:00.649905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:00.649905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:00.649905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:00.650340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:00.662987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 407/829] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:00.665119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.675205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 423/830] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:00.699345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.817279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea8f7-38f2-47a3-a669-108052facd2a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.842272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0607e4-3ade-42bd-9c27-14fdf8c5063d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.846577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:00.865704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.866385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.866666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.866936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.867713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:00.868022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.869410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e136b-82a8-4090-8f17-fb00cfa2a035 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.883556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 408/831] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:00.885013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459fe4a-287b-4b72-97a4-3fd01e5a3f50 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.885298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459fe4a-287b-4b72-97a4-3fd01e5a3f50 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.885571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459fe4a-287b-4b72-97a4-3fd01e5a3f50 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.891118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 409/832] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:00.891118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459fe4a-287b-4b72-97a4-3fd01e5a3f50 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.898259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea119abf-16b2-48ea-a463-80995f3eeee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.899436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea119abf-16b2-48ea-a463-80995f3eeee0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.899764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea119abf-16b2-48ea-a463-80995f3eeee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.900082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea119abf-16b2-48ea-a463-80995f3eeee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.922677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea119abf-16b2-48ea-a463-80995f3eeee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ea119abf-16b2-48ea-a463-80995f3eeee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:00.938343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:00.939389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:00.940258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:00.940912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 424/833] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:00.942951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.943463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.943972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.981262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:00.983414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:01.048729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459fe4a-287b-4b72-97a4-3fd01e5a3f50 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.052103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.099730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.100840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.101277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.101826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.115094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 425/834] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:01.117458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.117641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.119853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:01.152718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:01.155095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:01.157580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 410/835] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:01.161325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.223619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.288907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c95e2-d65b-4059-9fc0-0732b1427dee None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.323528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bded8-e667-4fc5-8f78-204870160bb5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.354170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.355263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.355715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.356128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.358509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:01.358994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.361537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.376729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 426/836] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:01.378155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.396938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.398163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.398573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.399002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.400077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:01.400535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.402388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.422217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 411/837] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:01.423616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.432086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.556001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5744-8d06-421a-b93e-36ed0cd86ba9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.591045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d1dab-0ae7-48ad-9346-a556dd5b5307 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.593387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.600103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.601115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.601526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.601983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.602985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.605069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:01.628492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:01.629591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:01.630450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 427/838] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:01.632085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.651706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.652870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.653434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.653717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.675725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.678560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:01.710693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:01.712482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:01.714234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:01.714234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:01.714234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:01.714234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:01.714234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 412/839] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:01.714778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.773164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.783183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae25f07-723b-4cda-a930-8a690dd1fe88 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.829707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.831047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.831378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.831647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/48141af292e144caa3ea38613964403e/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.846509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21de97-bc79-44f6-ae91-2c26f29629ae None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.853797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=48141af292e144caa3ea38613964403e, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.856611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.880480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 428/840] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:01 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/48141af292e144caa3ea38613964403e/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:01.881730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.897166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.899355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.899970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.900458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.901456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:01.902016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.908747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.927836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 413/841] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:01.929436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.970516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.063620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c4d2b-4e52-4002-965b-60a0750bb45c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.108220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.109357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.109878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.110374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.112028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:02.112485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.115172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.137945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 429/842] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.148304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.149962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9bd90-8afe-4cea-be42-38de43f9fb6b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.168824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.242336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.243547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.244039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.244520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.246573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:02.247151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.251229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30d01c-d579-40f7-976c-427a3670b877 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.263217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 414/843] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.354977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ae15b-7bd5-402f-a930-51b7d0585e52 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.356015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ae15b-7bd5-402f-a930-51b7d0585e52 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.356015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ae15b-7bd5-402f-a930-51b7d0585e52 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.360204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 415/844] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:02.366299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.366589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.366876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.385449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a773-ac4e-47a1-aac6-8d2b1708f9de None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.389242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:02.390716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:02.477234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119030c-a711-4508-a486-f18f91836294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.479590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119030c-a711-4508-a486-f18f91836294 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.480929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119030c-a711-4508-a486-f18f91836294 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.482876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119030c-a711-4508-a486-f18f91836294 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.487255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 416/845] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:02.505466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119030c-a711-4508-a486-f18f91836294 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.510057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.511420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119030c-a711-4508-a486-f18f91836294 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1119030c-a711-4508-a486-f18f91836294 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:02.529894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:02.530936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:02.531948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:02.533776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 430/846] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:02.539090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.540895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119030c-a711-4508-a486-f18f91836294 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.658960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.737791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55dd31-d0e0-4943-9f63-a63a9d4556da None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.742391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119030c-a711-4508-a486-f18f91836294 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.809810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.811641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.812257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.812874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.813045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.814112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.814438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.814607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.815045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.816657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:02.817179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.817575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.819819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:02.843617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:02.847145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:02.850369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 417/847] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:02.857749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.875285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 431/848] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.877425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.004273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.047919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d20c-8995-4ee1-9f35-ffbb9cbe1de6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.050763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8ad3-cc29-43b0-9704-94dfe39e21ae None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.082221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.095752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.096592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.096983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.097371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.121038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.122247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.122891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.123335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.124739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:03.125160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.127003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.139222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 418/849] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:03.140911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.257350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.260540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.363131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a87ff-96a7-4118-8db5-c17e558e0770 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.447713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.449775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.451218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.452560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.454233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:03.455624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.458425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.479675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 419/850] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:03.497305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 432/851] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:03.508014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.515650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.677963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51429-95e6-49ae-a341-2949be96e299 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.720483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.721303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.721588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.722291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.737543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.737543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:03.757051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:03.758126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:03.759232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:03.759232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:03.759232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:03.759232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:03.759232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 420/852] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:03.766707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.803440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.921166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330e57-30b8-483d-adcd-c8ae43954110 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.961773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.962867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.963397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.963886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.964913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:03.965436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.967642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc98eef-caad-44a3-acdc-a17685e1ae5e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.981317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 433/853] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:03.986890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.053452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.105269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae90ea-d7b4-46b7-8584-3a7ae728fe73 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.145394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.146454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.146742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.147018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.155891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.157778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.166001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b6541-4882-4fd0-969d-4f1284963c6d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.166292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b6541-4882-4fd0-969d-4f1284963c6d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.166573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b6541-4882-4fd0-969d-4f1284963c6d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.168289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 434/854] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:04.173879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.174195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.174447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:04.175121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:04.175992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:04.177106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 421/855] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:04.184170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.201370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:04.202776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:04.311165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 435/856] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:04.324497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30429b30-29f1-46c4-93be-1d835ffbf7be None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.331750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.345001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.350198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.365820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.366514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.366799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.367109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.367949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:04.368257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.369771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.388480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 422/857] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:04.389978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.508337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c8091-0dd0-4494-9591-333a13b7b02d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.553547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.554948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.555571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.556052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.568473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:04.575304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d657a0-04fa-4f06-8bd2-a493f05a6498 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:04.577695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:04.578637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:04.579531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:04.587010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 436/858] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:04.588275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.655246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.655919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.656203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.656479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.660259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=08ed51a3aab34ac0bf5791fa3307a79d) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:04.660259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.660259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.675090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 423/859] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/role_assignments?user.id=08ed51a3aab34ac0bf5791fa3307a79d&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:04.704816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.743572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d4f13-dfc3-4270-9a2f-fac1beee2c50 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.785848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.786618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.786809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.787163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.787933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:04.788242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.789728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26098b8a-4725-4a88-9a00-1d536d8e0b8b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.799931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 437/860] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:04.808269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.864706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.869640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebbbc2-1587-4daf-aa1e-fc0177682621 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.869940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebbbc2-1587-4daf-aa1e-fc0177682621 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.870217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebbbc2-1587-4daf-aa1e-fc0177682621 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.873681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 438/861] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:04.878567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.878872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.879175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.899243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:04.900775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:04.949641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce41e1-5418-4433-9d01-03480d6e74c9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.994145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.995511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.995943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.996362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.005723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 439/862] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:05.009627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.011277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.030752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:05.030958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.031986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.033069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:05.033069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:05.033069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:05.033069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:05.035462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 424/863] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.040622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.119974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.187027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6da112-7e1b-4a9e-9f93-b9ce17f1e68c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.196092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.208387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9283a-b8ab-4ad8-98d7-775c2f34d851 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.250870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.251938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.252380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.252952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.253762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:05.254487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.255745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.265730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.267502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.268344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.268805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.269868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.270860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 425/864] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:05.271995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.276456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.296670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 440/865] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:05 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:05.462604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d58802-da00-491c-857a-5960b4f80ef6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.512683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.513588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.513878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.514164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.522538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.524048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:05.540621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.541808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:05.542775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 426/866] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.548611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.668788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32579f12-6f8a-49d0-a16b-919a7daeaaee None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.714008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.714621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.714908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.715236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.716274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:05.716662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.718141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92a5ab-39d5-4e09-add7-9956217ea972 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.720099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.739639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 441/867] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:05.745284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.759435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.876453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad739e-c7fa-4dae-8f15-f9ad86994a29 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.889495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badaeb-c5c1-48c4-a285-c14ade4a4123 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.890030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badaeb-c5c1-48c4-a285-c14ade4a4123 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.890122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badaeb-c5c1-48c4-a285-c14ade4a4123 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.891652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 442/868] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:05.901473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.901905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.902290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.915995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.916668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.916954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.917233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.922233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:05.923634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:06.003211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.004971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.007371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 443/869] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:06.018764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.028704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.164927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 427/870] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:06.165376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1068c-0d9d-454b-8fcf-48f1a0f87034 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.166409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.185830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.225603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.226656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.227240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.227778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.229706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.234499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.264811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 444/871] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:06 2022] POST /identity/v3/services => generated 269 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:06.322802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959711a-a97d-4bbf-b691-f23a6ff8f9a9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.336699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.380150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.380963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.381244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.381521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.395568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.397317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:06.418949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:06.420474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:06.421372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:06.421372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:06.421372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:06.421372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:06.422025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 428/872] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:06.428252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.558954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dadc5f-15a5-49ff-9a55-0df3d83a0c1a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.602379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.603536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.604065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.604554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.605629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:06.606172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.609125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.624697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 445/873] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:06.632319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.820966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05203b09-6d09-42de-85c5-991af1d599f1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.876205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.877636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.877927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.878206 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.885999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.887516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ff1682c7-6931-47e8-a005-9f5938bfe834 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:06.905331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:06.906301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:06.907257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 429/874] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:06.912947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.954343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.033152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f7dcd-6b0f-4713-bb3a-e7516723c8e6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.077273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.078393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.078897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.079428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.081045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.081627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.083170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bce2b9-ec08-448f-8e40-c6a7f9d046a4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.083485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bce2b9-ec08-448f-8e40-c6a7f9d046a4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.083772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bce2b9-ec08-448f-8e40-c6a7f9d046a4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.083944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.086885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 430/875] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:07.092305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.092695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.092905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.112582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:07.113897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:07.117018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 446/876] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.123720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.128545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.215135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 431/877] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:07.236097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.327559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.342269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6a403-f9b5-4d1f-9383-7b3bb40ee4f1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.368885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f815f-6d2a-4c2e-9ebb-65224e0d37ac None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.410971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.411789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.411974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.412134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.412416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.413101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.414096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.414519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.414962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.415752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.417334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=48141af292e144caa3ea38613964403e) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.418102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.422639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587899f-7799-4d4e-ab61-d6a827ab3a0d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:07.428837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:07.430506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:07.432533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:07.435757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 432/878] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:07.440574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.447250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 447/879] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/role_assignments?user.id=48141af292e144caa3ea38613964403e&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.466769 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.588212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fe432-ff07-438b-84fe-16b82be4413d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.588468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fe432-ff07-438b-84fe-16b82be4413d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.588812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fe432-ff07-438b-84fe-16b82be4413d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.590374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 448/880] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:07.596696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.596977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.597282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.614685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7e90-af57-4f73-b5fd-c286464324a4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.616587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:07.618007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:07.657401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.658225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.658471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.658748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.662460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.662460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.662460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.668070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 433/881] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.674466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.730478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 449/882] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:07.774577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.780013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.827865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc64ab-293e-40de-a762-b2531b86cf94 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.901045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.903196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.903196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.903196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.905006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.905533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.908063 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.923292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 434/883] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.942163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.964674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0e3fe-8961-4a4e-bdd0-c443a7c95301 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.033720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.035535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.036256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.036930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.038367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.041323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:08.065506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:08.067729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:08.069609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:08.074080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 450/884] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:08.084897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.290336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad6230-e01a-4b35-aabd-76f433236068 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.354286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.355439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.355938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.356464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.358291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:08.358840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.361442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad6b20-41a8-4507-9759-6d50a044b68b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.371015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 435/885] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:08.380240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.577658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df49fe2-f8e6-4753-96df-5a457777595f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.612650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077505-f391-4a54-a671-2311cae0f925 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.612916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077505-f391-4a54-a671-2311cae0f925 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.613274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077505-f391-4a54-a671-2311cae0f925 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.614974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 436/886] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:08.620723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.621113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.621354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.629381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.647597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.648715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.649204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.649712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.650961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:08.651533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.652693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:08.654423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.654647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:08.665712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 451/887] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:08.674699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.711519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.760205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 437/888] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:08.800490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.881681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021a42-ad29-4a50-836e-a0a17f24e9ad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.955607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.956733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.957228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.957716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.959256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] RBAC: Adding query filter params (interface=public, service_id=61548c5f94424e1b9db5635a8c629143, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:08.959818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.962460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.979462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 452/889] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/endpoints?service_id=61548c5f94424e1b9db5635a8c629143&interface=public®ion_id=RegionOne => generated 194 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:08.982325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.006089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642ec64-bd95-4c7c-b5f5-4ba940269222 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.080618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.089738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.091944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.092799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.093567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.095137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.097885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:09.123175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:09.124185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:09.125028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:09.125822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 438/890] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:09.136180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.156232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.181311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6ffcf-8d49-4f0e-8b5a-d2b26b267693 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.241156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.242210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.242595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.242968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.243802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.245658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.257788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 453/891] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/services => generated 1268 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:20:09.358821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84726167-aac5-4190-8589-d4b27de24a27 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.428066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.429570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.430284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.430981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.432696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:09.433477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.436945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.448900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 439/892] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:09.458944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.608187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b374b-9fc9-4750-bed6-5cc2f0deab49 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.648764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.649848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.650333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.650806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.651805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:09.652330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.654518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.661095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 454/893] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:09.671002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.798358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.878540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae415f-70c0-4f9e-9c77-d06a7f0870ea None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.921889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.922972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.923522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.924012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.925060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] RBAC: Adding query filter params (interface=public, service_id=7e4994f4d686473e871179a6ec0a8729, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:09.925598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.927267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44389ba-09cc-496a-af0a-12a2553c9671 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.937028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 440/894] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/endpoints?service_id=7e4994f4d686473e871179a6ec0a8729&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:09.945467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.966964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:10.086567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcfbf5-16fa-45de-b273-e93671600c59 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.126045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715dee0-4446-4947-8712-837dff42468d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.126732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dee0-4446-4947-8712-837dff42468d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.127014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dee0-4446-4947-8712-837dff42468d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.127332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715dee0-4446-4947-8712-837dff42468d None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.128029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715dee0-4446-4947-8712-837dff42468d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:10.129436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715dee0-4446-4947-8712-837dff42468d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:10.136452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 455/895] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/services => generated 1268 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:20:10.532454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5702-05c3-42b9-bd82-019e36c63c22 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.532850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5702-05c3-42b9-bd82-019e36c63c22 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.533052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5702-05c3-42b9-bd82-019e36c63c22 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.534766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 441/896] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:10.540554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c917e30-7147-4ecd-9593-8d1122335a0f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.540852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c917e30-7147-4ecd-9593-8d1122335a0f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.541141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c917e30-7147-4ecd-9593-8d1122335a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.560068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8c917e30-7147-4ecd-9593-8d1122335a0f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:10.561294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8c917e30-7147-4ecd-9593-8d1122335a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:10.678838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 456/897] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:10.707993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c5702-05c3-42b9-bd82-019e36c63c22 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:10.861851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c5702-05c3-42b9-bd82-019e36c63c22 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.910194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:10.928740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb613c-9df7-4a6f-bff0-a40dc733d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.929423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb613c-9df7-4a6f-bff0-a40dc733d1bb None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.929711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb613c-9df7-4a6f-bff0-a40dc733d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.929993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb613c-9df7-4a6f-bff0-a40dc733d1bb None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.930744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb613c-9df7-4a6f-bff0-a40dc733d1bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:10.932184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb613c-9df7-4a6f-bff0-a40dc733d1bb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:10.955356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 442/898] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:10 2022] POST /identity/v3/services => generated 241 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:10.962654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76a4fc-5482-429c-b996-dbd71e94b8d9 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.963689 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76a4fc-5482-429c-b996-dbd71e94b8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.964457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76a4fc-5482-429c-b996-dbd71e94b8d9 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.975187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 457/899] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:10.978488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.978817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.979102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.998788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:11.000127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:11.138655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 443/900] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:11.162353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76a4fc-5482-429c-b996-dbd71e94b8d9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.166955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.361158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76a4fc-5482-429c-b996-dbd71e94b8d9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.428036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.429335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.429846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.430351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.450894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:11.468206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:11.470024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:11.471704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 458/901] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:11.483807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.681581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3100f-b875-4c89-9416-7ad58f8685f8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.709479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.757716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.758875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.759427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.759938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.761401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:11.761968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:11.765805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d98ab-717d-408e-aebd-6c25be463ff8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:11.782220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 444/902] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:11.791971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.917506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.934175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9abdc9-962c-4af1-a163-969c71e318c1 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.935815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9abdc9-962c-4af1-a163-969c71e318c1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.936331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9abdc9-962c-4af1-a163-969c71e318c1 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.942261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 445/903] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:11.944395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.944915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.945400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.963866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:11.965562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:11.994992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c6bb1-8405-45d7-bfe4-61f3ed98859a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.047938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 446/904] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:12.058774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.060687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.061197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.061695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.069514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.087643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.090791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:12.132323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:12.134279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:12.136327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:12.136327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:12.136327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:12.136327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:12.140585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 459/905] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:12.149046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.188723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51f73f-d4e5-46b3-ae57-0fff7c26aa44 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.197983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.246514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.247976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.248417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.248931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.250835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.253859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:12.282759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:12.284860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:12.292270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 447/906] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:12.298711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.323526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cdd67-68ec-425d-b92f-1d0e2ee61253 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.342847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.370062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.370744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.371030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.371344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.372725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:12.373085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.374325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.390584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 460/907] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:12.396886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.471218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe5e9-903f-4f6c-bd37-9de5ca621404 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.526314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.527452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.528043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.528866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.530102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:12.530415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.532153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d2d10-3c44-4969-92c1-0e8ce65bb0a9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.539193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 448/908] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:12.540187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d7f18-8390-4357-b29f-dde02d04906b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.544630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e4ad1-0b05-4374-9f95-15860fd2817a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.544922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e4ad1-0b05-4374-9f95-15860fd2817a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.545183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e4ad1-0b05-4374-9f95-15860fd2817a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.546579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 449/909] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:12.547338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e4ad1-0b05-4374-9f95-15860fd2817a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.581392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.607675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be000009-8951-47bc-817e-904c88daffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.609112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be000009-8951-47bc-817e-904c88daffff None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.609581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be000009-8951-47bc-817e-904c88daffff None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.610083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be000009-8951-47bc-817e-904c88daffff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.625319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be000009-8951-47bc-817e-904c88daffff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.628916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be000009-8951-47bc-817e-904c88daffff None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-be000009-8951-47bc-817e-904c88daffff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:12.651194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:12.652520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 461/910] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.655357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.656180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.680518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e4ad1-0b05-4374-9f95-15860fd2817a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.688981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:12.691093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:12.726070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.726660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.726945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.727252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.727971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:12.728283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.729712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.742559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 450/911] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:12.742559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.746130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.808442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 462/912] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:12.809804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.870476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.899632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955790c-4277-4516-adbb-58b29b6c5e37 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.949676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.951316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88370c-f0cf-4051-8432-a0b366d07004 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.951665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.951665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.951950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.953187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:12.953820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.955979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.978243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 451/913] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:12.978822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.011204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.013179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.013636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.014216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.016129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.016129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.044910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 463/914] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:13.046782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.092091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.163414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01c38c-196c-49bc-90e7-c6f190a3e84e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.229564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c268b-14ef-4bf1-a268-44b606d63068 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.233670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.234290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.234711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.235205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.236223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.238380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:13.258418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:13.260204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:13.261761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 452/915] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:13.275237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.282795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.283854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.284310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.284802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.313060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.385353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.388342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.426030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca630f-82b1-43d1-ae63-1aadc011c552 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.431509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.477205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.478279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.478774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.479207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.480147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.480544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.482808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.493013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 453/916] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.494297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.528535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 464/917] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:13.535097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.629383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.663913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12701c67-ef50-40f4-a0b1-3f78ea13bba3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.675555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09881c74-79b1-4dab-81b1-4d81b4ffd7c6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.696030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.714098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.714782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.715125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.715369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.716802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.717126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.718587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af24d-6e26-466e-88b5-fd8e87ef2e37 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.719314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.720202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.720558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.720917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.724939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 454/918] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.725964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360e21-856a-4544-bb33-06db5dd68cef None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.726255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360e21-856a-4544-bb33-06db5dd68cef None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.726533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360e21-856a-4544-bb33-06db5dd68cef None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.728081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 455/919] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:13.733460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360e21-856a-4544-bb33-06db5dd68cef None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.740201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.741700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fc7b9b16-d325-48c7-8043-537cf28fe037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:13.761450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:13.762862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:13.763810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:13.763810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:13.763810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:13.763810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:13.770628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 465/920] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:13.772638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.772920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.773201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.796699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:13.798038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:13.871465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360e21-856a-4544-bb33-06db5dd68cef None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.907094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 466/921] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:13.907094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.922482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.923582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.923942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.924289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.925185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.926734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.946611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.959618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 456/922] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:13.960532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.092728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.100412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e1627-2ded-44f7-9ac2-68fd2dd47c93 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.119975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0b899-7160-4903-b4dc-47b70575d155 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.169792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.170418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.170818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.171226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.172039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.172419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.173888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.179223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.180592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.180956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.181486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.182262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.184001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.187796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 467/923] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.196234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:14.199312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:14.201442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:14.202524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 457/924] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:14.208094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.299478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.346648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d887-7ca0-47f4-aecf-e72fa914658c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.361919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf359-5852-4858-bd27-c1efcc9a416d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.385593 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.386520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.386902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.387846 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.388857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.389262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.390760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.396960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 458/925] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:14.403745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.408476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.409803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.410269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.410721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.419947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.421817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:14.439261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:14.440423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:14.441457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:14.448430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 468/926] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:14.450411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.473658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.570072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc292570-7b2d-4c38-9eb0-39ad01cbaf10 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.618184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.629475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007fc3-a114-45db-b0f0-f24489904867 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.639547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.640912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.641613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.642274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.643745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.644459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.648379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.664109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 459/927] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.672959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.695173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.696792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.697619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.698410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.700439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.701344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.704655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.728413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 469/928] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:14.735814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.845612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.859504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65352b0c-ded4-4aae-8d11-99a6ea3ab256 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.875978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c74a1-97f2-4f24-9e9a-9116c237e037 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.930702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.931456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.931750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.932030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.932910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] RBAC: Adding query filter params (role.id=6fd3c8340d12478680af497dd225c736, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=48141af292e144caa3ea38613964403e) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.933226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.933659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.935618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.936413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.937215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.937994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.939914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] RBAC: Adding query filter params (interface=public, service_id=789562f90d7a4e0d913ccd67f6152788, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.940788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.943719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894042-18ca-4626-9d8c-5e66227e8b5b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.947196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 470/929] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/role_assignments?user.id=48141af292e144caa3ea38613964403e&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=6fd3c8340d12478680af497dd225c736 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.957801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 460/930] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/endpoints?service_id=789562f90d7a4e0d913ccd67f6152788&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:14.968305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.177467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000e424-f833-455f-a5bc-4dbc3a375242 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.216098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7fa88-d017-47f1-8b4a-6679a58a4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.216781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7fa88-d017-47f1-8b4a-6679a58a4a1f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.217058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7fa88-d017-47f1-8b4a-6679a58a4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.217326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7fa88-d017-47f1-8b4a-6679a58a4a1f None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.218022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7fa88-d017-47f1-8b4a-6679a58a4a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.219601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7fa88-d017-47f1-8b4a-6679a58a4a1f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.226607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 471/931] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/services => generated 1497 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:20:15.670672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbda26-0701-4116-8f30-252f0872e11e None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.671533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbda26-0701-4116-8f30-252f0872e11e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.671682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbda26-0701-4116-8f30-252f0872e11e None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.674425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 461/932] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:15.681351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.682075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.682671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.712993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.723845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:15.725907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:15.807527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 472/933] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:15.847959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbda26-0701-4116-8f30-252f0872e11e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.935219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.000914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbda26-0701-4116-8f30-252f0872e11e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.039990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.041047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.041712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.042193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.043157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.045401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.047032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING keystone.server.flask.common [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 22:20:16.068589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.sql.core [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:20:16.068589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 22:20:16.068589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [parameters: {'id': '89709fc4ff2c47f29db5ffe4952e8d46', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 22:20:16.068589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80276) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2b2625a2-f54c-477f-8d14-ba1bda0739cf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 89709fc4ff2c47f29db5ffe4952e8d46.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 89709fc4ff2c47f29db5ffe4952e8d46. Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:16.077869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 22:20:16.078780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application yield resource Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise self.value Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application session.commit() Jul 21 22:20:16.079851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:20:16.080782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:20:16.081725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:16.082631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [parameters: {'id': '89709fc4ff2c47f29db5ffe4952e8d46', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.085164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:16.087093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 89709fc4ff2c47f29db5ffe4952e8d46. Jul 21 22:20:16.089354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:16.093416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 462/934] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Jul 21 22:20:15 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:16.101415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.245070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60502-1f13-4acc-8a09-3a5505592b90 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.245338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60502-1f13-4acc-8a09-3a5505592b90 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.245594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60502-1f13-4acc-8a09-3a5505592b90 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.255261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 463/935] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:16.259902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.260521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.261114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.283169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:16.284854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:16.297791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdae9d-cfbb-43e3-b2b4-ccaab5754795 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.367095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 464/936] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:16.368347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.369174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.369457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.369726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.373234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.379622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.381098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:16.396733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:16.398427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:16.400367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 473/937] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:16.406090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.420872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.451935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.546018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d89ddd-357c-4ecc-8a8d-e1c065c96834 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.594304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.594996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.595316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.595600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.596381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:16.596690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.598280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.599949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563711cd-4f5e-40ae-bea4-c613d16cfc75 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.623334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 465/938] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:16.667851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.668742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.669023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.669293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.669959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.671435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:16.685987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:16.688139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:16.689913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 474/939] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:16.703832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.904257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.911721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14cf00-bb89-4bcd-a830-5ab65abb74cc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.977992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.979229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.979738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.980242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.982156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] RBAC: Adding query filter params (name=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:16.982725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.985603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0958-c7a8-417a-b8c3-fe23c3fe8dfe None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.999641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 466/940] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:17.004809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.131470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:17.214443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25ce3-8892-4bf5-9592-98090ae86535 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.285503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.286657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.287194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.287690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.289035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:17.289596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.292260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99573-f63e-4f1b-83f5-9f589bd8a275 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.301980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 475/941] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:17.637956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e10d9-7603-4b04-9c61-25b58b2d5449 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.638349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e10d9-7603-4b04-9c61-25b58b2d5449 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.638622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e10d9-7603-4b04-9c61-25b58b2d5449 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.640428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 467/942] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:17.646437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.646747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.646994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.683069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:17.686285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:17.818779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 476/943] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:17.844886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e10d9-7603-4b04-9c61-25b58b2d5449 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.983988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e10d9-7603-4b04-9c61-25b58b2d5449 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.027043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4d767-5ef2-4842-9b0d-696aac3f23b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.028734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4d767-5ef2-4842-9b0d-696aac3f23b9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.029198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4d767-5ef2-4842-9b0d-696aac3f23b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.029634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4d767-5ef2-4842-9b0d-696aac3f23b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.042113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4d767-5ef2-4842-9b0d-696aac3f23b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-20e4d767-5ef2-4842-9b0d-696aac3f23b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:18.050837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:18.052046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:18.052979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:18.055479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 468/944] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:18.064693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.086703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446896bf-1cb2-4811-9b7d-f201b99c4707 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.087010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446896bf-1cb2-4811-9b7d-f201b99c4707 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.087329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446896bf-1cb2-4811-9b7d-f201b99c4707 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.089012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 469/945] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:18.095167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.095461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.095744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.114267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:18.115717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:18.136953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.172758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.216041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 470/946] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:18.239115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.269035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc065-8e43-4cc7-bf69-1f41cf760f36 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.338160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.341009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.341849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.342508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.344256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.345095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.348291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.371094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73183bd7-3d39-449a-a19c-6edb9f731438 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.372123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 477/947] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:18.377575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.411787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.412924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.413216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.413498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.414181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.415690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:18.431306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:18.433479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:18.436729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:18.438650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 471/948] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:18.439849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.474031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.545271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0356-faa3-4776-a3b9-e97c7cd2162d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.567239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.590081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.590940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.591452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.591747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.610994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.614296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.626359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428fbb-7eef-453f-af99-812566fba94b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:18.652195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:18.654523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:18.656649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:18.656649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:18.656649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:18.656649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:18.667151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.667938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.668146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.669660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 478/949] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:18.669864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.670639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.670964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.672438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.674810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.683307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 472/950] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:18.687168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.797963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.826335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73bcfc-4bb9-4ef0-81a7-d4b8490062b4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.866182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.867432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.867432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.867432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.868196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.868514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.869957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.881215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 473/951] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:18.882244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.909058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999b116-1409-453c-816c-b4303bc2c2f5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.976188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.976866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.977154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.977429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.978746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.979094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.980511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65afab-8bfe-4039-b03f-2de377c8e7dd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.995154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 479/952] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:18.996556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79d308-f30e-4070-8670-cac456cbffaf None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.997011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79d308-f30e-4070-8670-cac456cbffaf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.997801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79d308-f30e-4070-8670-cac456cbffaf None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.999199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 480/953] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:19.001370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79d308-f30e-4070-8670-cac456cbffaf None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.027004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb0b64-a075-48f8-8f3d-71bd2e9632b1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.037377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.087356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef703e-181e-4332-9ea9-d6bd8b0ce539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.088240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef703e-181e-4332-9ea9-d6bd8b0ce539 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.088705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef703e-181e-4332-9ea9-d6bd8b0ce539 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.089121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef703e-181e-4332-9ea9-d6bd8b0ce539 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.090024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef703e-181e-4332-9ea9-d6bd8b0ce539 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.092454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef703e-181e-4332-9ea9-d6bd8b0ce539 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.110509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.127508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 474/954] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:19.133976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.133976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.133976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.152401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:19.154707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:19.201974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79d308-f30e-4070-8670-cac456cbffaf None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.249395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 475/955] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:19.268728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.269803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.270096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.270373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.281461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.283019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.295797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:19.303096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:19.304936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:19.308331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 481/956] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:19.317067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.454241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.476775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f9ac-26eb-4563-9883-46c67d4e1470 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.500874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1518fd8-23d6-4b8f-91b2-0c649e3e7462 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.552394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.553634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.554148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.554644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.556401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:19.556979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.560555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.570527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.576806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.578474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.581381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.582233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.583970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.586917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.596011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 482/957] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:19.604642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.609329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.sql.core [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:20:19.609329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 22:20:19.609329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [parameters: {'user_id': '0ad0bfb2ca9d46d083ca10ef3f60d87a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 22:20:19.609329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80276) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:19.617573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:19.618540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application yield resource Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise self.value Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application session.commit() Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:20:19.619588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:20:19.620506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:19.621574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:19.622889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [parameters: {'user_id': '0ad0bfb2ca9d46d083ca10ef3f60d87a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.625982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 22:20:19.627975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 476/958] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jul 21 22:20:19 2022] POST /identity/v3/users => generated 160 bytes in 325 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:19.631767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.742921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42412-0fe1-4270-8cd2-78021c6795db None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.756862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eedb4-7e45-473a-8856-6f5c477babc9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.785086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.786295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.786833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.787362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.797194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.798009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.798295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.798574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.801510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.803308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.805568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.807920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.809922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:19.825877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:19.828015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:19.830071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:19.831392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:19.835255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:19.836224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:19.836224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:19.836224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:19.836224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:19.836473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 477/959] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:19.838517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 483/960] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:19.839760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.844424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.901049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.959095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677e427-9513-4f29-adf7-b1df1fb807b9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.974602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636266-e4b8-4082-97d3-e3372894464d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.998038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.998625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.998914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.999231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.000059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:20.000375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.001839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7974e9-0e82-4362-ad29-14b16c421c66 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.019106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.020281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.021060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.021554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.022548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:20.023123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.024744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.045258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 484/962] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:20.046592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.050690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 478/962] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:20.070010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f425-5c7d-41c9-b3ac-198054dd5b74 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.070344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f425-5c7d-41c9-b3ac-198054dd5b74 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.070582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f425-5c7d-41c9-b3ac-198054dd5b74 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.072352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 479/963] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:20.080739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.081086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.081419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.100784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:20.102570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:20.123465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.182280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f70d5-862d-47cc-a63f-f6bd54ae72d1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.195812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 480/964] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:20.213248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.226608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.227964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.228424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.228851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.238629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.242000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:20.256500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:20.257433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:20.258423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 485/965] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:20.264756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.272087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.363667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322e8e-d4ad-4f0c-b9c2-b9a460431a1a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.403029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.403738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.404016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.404289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.404620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094c454-64cc-44a5-9c0f-c4861e525209 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.405036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.406425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.422605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.427299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 481/966] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:20 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:20.457664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.458395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.458645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.458984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.459877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:20.460200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.461739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdfd6f-2e37-4adb-950c-79f17f9a6a6f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.477046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 486/967] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:20.484149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.601441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.605382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0289d58-513a-4008-84a6-098f3d57007b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.649231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363285fa-775b-4d06-bd9d-67c7fcd26662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.650139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363285fa-775b-4d06-bd9d-67c7fcd26662 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.650425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363285fa-775b-4d06-bd9d-67c7fcd26662 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.650703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363285fa-775b-4d06-bd9d-67c7fcd26662 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/48141af292e144caa3ea38613964403e/roles/6fd3c8340d12478680af497dd225c736` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.652990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363285fa-775b-4d06-bd9d-67c7fcd26662 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=48141af292e144caa3ea38613964403e, role_id=6fd3c8340d12478680af497dd225c736)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.654434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363285fa-775b-4d06-bd9d-67c7fcd26662 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.679135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 482/968] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:20 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/48141af292e144caa3ea38613964403e/roles/6fd3c8340d12478680af497dd225c736 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:21.927113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1918-7008-4d2d-83d7-ab71496a82e6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.927643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1918-7008-4d2d-83d7-ab71496a82e6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.927643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1918-7008-4d2d-83d7-ab71496a82e6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.928992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 487/969] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:21.935047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.935585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.935660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.953381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:21.954584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:22.037098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 483/970] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:22.072332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1918-7008-4d2d-83d7-ab71496a82e6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.206353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1918-7008-4d2d-83d7-ab71496a82e6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.255971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.257398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.257912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.258421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.259656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.262981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:22.287815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:22.295656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:22.297666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 488/971] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:22.305616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.422501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.439474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f80f33-ebfb-4c36-8164-3e8f2cb8dfa0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.483208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.483796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.484081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.484360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.485094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.485402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.486830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.493296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 484/972] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:22.499756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.549855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.682094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca09d11-96cb-4226-8824-4ec7cc2d250a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.754542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.755655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.756137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.756614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.758034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.758600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.760200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.767672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 489/973] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:22.775636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.928463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5372d2-9f21-42c2-a276-673d9084e08e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.970281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.971356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.971356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.971606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.972320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] RBAC: Adding query filter params (interface=admin, service_id=789562f90d7a4e0d913ccd67f6152788, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.972630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.974021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9563e6-d9bd-4bbf-b131-9fafaf77602c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.981257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 485/974] 10.222.0.58 () {56 vars in 1249 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/endpoints?service_id=789562f90d7a4e0d913ccd67f6152788&interface=admin®ion_id=RegionOne => generated 193 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:22.986788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.023326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b8672-2135-4f61-b219-f71195206cd6 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.023582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b8672-2135-4f61-b219-f71195206cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.023768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b8672-2135-4f61-b219-f71195206cd6 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.025402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 486/975] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:23.031562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.031970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.032131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.055945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:23.057830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:23.082385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.186848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.187155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 487/976] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:23.198385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb064c-dade-4705-91bb-eab898116659 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.210658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.271402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075eef44-28a7-4b38-92c6-883bb111111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.272561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eef44-28a7-4b38-92c6-883bb111111d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.273081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eef44-28a7-4b38-92c6-883bb111111d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.273718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eef44-28a7-4b38-92c6-883bb111111d None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.274997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075eef44-28a7-4b38-92c6-883bb111111d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.278393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075eef44-28a7-4b38-92c6-883bb111111d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.291714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 490/977] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/services => generated 1727 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:20:23.323979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c857e-8ab6-4dba-9f4c-d1031a13234a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.325388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c857e-8ab6-4dba-9f4c-d1031a13234a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.325556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c857e-8ab6-4dba-9f4c-d1031a13234a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.328289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 491/978] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:23.335495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.336012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.336519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.368265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:23.370328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:23.439581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477e1db-00c5-45fb-b121-ddd09f171999 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.471219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 492/979] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:23.497157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.502592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.503489 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.503858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.504146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.504827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.506239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:23.519155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:23.521687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:23.523601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 488/980] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:23.529547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.547566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.663532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.683003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5aef1-284b-4d8b-9027-41078650acba None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.688163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276233c5-f1d6-4ef5-8344-91f73a0b0037 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.750411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.751884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.752029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.752399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.752913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.753146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.753628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.754108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.755481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.756032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.758603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.769036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 489/981] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:23.776169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:23.778156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:23.793946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:23.796508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:23.798256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 493/982] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:23.805299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.905344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.935832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b5090-20bc-4175-a7c8-2a0843389d26 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.938671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114c7f-6e8c-4733-a2d6-bc0232ff8037 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.988841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.989896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.990291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.990552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.991252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.991400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.991531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.991837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.992122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.992531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.992686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.992929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.994251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.994436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.006586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 490/983] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:24.007307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:24.012272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 494/984] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:24.021417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:24.044205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:24.186317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d24d30-fa09-4a22-8341-e71ee3dc77dd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:24.211307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7306a-0b67-4667-b659-c5873424d1e2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:24.249069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:24.250117 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:24.250573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:24.251021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:24.252214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] RBAC: Adding query filter params (interface=public, service_id=09a14a4e6a334acea6a2057e54be9014, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:24.252732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:24.255114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.271113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 491/985] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:24 2022] GET /identity/v3/endpoints?service_id=09a14a4e6a334acea6a2057e54be9014&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:24.274213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:24.281059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:24.282089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:24.282445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:24.282780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:24.283512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:24.297620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:24.301903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:24.329596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:24.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:24.334086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:24.334086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:24.334086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:24.334086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:24.334086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 495/986] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:24.337331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:24.419194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:24.465213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760ef4-d980-4bf4-9bc9-ea40478bb8be None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:24.489126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1907-82fd-4095-a7fb-1cfd5a127df9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:24.507619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:24.508342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:24.508652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:24.508983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:24.510002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:24.510352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:24.512483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.528362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 496/987] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:24.534790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:24.551677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:24.552395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:24.552838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:24.553292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:24.554102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:24.555659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.567103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 492/988] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:24 2022] GET /identity/v3/services => generated 1727 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:20:24.704310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b990-ffe3-4057-a806-9033800f3f33 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:24.759261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:24.784316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:24.785727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:24.786226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:24.786749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:24.800701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:24.803400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:24.834504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:24.835696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:24.837212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:24.838303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 497/989] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:24.845994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:24.941420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:24.988060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384189b-bdf3-48a8-9364-21a2cce05a51 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:25.029655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:25.030737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.030885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.031250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.031959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:25.033313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:25.034789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:25.064706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 493/990] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:25.070561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:25.284935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1872-da15-40bd-bb55-7a1f3314fe8f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:25.361616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04176a65-0d4c-44d5-91ca-7cc384a6e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:25.362759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04176a65-0d4c-44d5-91ca-7cc384a6e34a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.363270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04176a65-0d4c-44d5-91ca-7cc384a6e34a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.363758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04176a65-0d4c-44d5-91ca-7cc384a6e34a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.452371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:25.470383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04176a65-0d4c-44d5-91ca-7cc384a6e34a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:25.472052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04176a65-0d4c-44d5-91ca-7cc384a6e34a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:25.602040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 498/991] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:25 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:25.609378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:25.786649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab31be6-fc93-4d06-a145-5779b3096f13 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:25.797807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff85cd-f15f-4127-bfeb-54d4cd1b13b7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.798916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff85cd-f15f-4127-bfeb-54d4cd1b13b7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.799408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff85cd-f15f-4127-bfeb-54d4cd1b13b7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.802371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 499/992] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:25.813794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.814259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.814716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.831086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:25.842611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:25.844579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:25.847605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:25.848176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.848625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.849077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.875159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:25.876654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:25.904673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:25.906449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:25.915945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:25.915945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:25.915945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:25.915945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:25.915945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 494/993] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:25.915945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:25.951592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 500/994] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:25.978463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.001808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.063270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efc0a4-4242-49ed-9ffa-d7b289b3b25b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.103678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a1569-fe2a-4988-9f80-fe8c1a652db6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.107858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.108565 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.108852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.109131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.109893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.110205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.111971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.126190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 495/995] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:26.131745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.151096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.152055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.152864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.153146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.153829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.155317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:26.168888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:26.170137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:26.171011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 501/996] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:26.176832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.189140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.267328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d63d6a-ceba-478e-88c5-46368cfa02f6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.284379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.305922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.306729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.307015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.307326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.307440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf15d46-7108-4ca8-8e3b-5cdbe78408c6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.316557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.318051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:26.334492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:26.337064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:26.339466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 496/997] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:26.342678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.365646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.366472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.366796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.367189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.368394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.368394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.369687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.376640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 502/998] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:26.382242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.462870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.467797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17208be2-ba7a-4602-ae97-4489cbee1d99 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.511166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.512345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.512636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab06823-45cb-4212-88f3-96fc302de11b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.512859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.513451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.514961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.515573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.517608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.549437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 497/999] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:26.551561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.552991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.553677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.553960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.554240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.554952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.555291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.556925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.564351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 503/1000] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:26.570752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.591324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.731514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1b6e-610e-4e16-b868-74dc53375bc9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.738933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a530c-e1e6-45e8-8f85-ca87698297df None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.793361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.793879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467a5d0-7cca-429f-a142-e4b5d10a75e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.794613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.794721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467a5d0-7cca-429f-a142-e4b5d10a75e5 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.795019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.795157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467a5d0-7cca-429f-a142-e4b5d10a75e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.795543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467a5d0-7cca-429f-a142-e4b5d10a75e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.795639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.796448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467a5d0-7cca-429f-a142-e4b5d10a75e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.797157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] RBAC: Adding query filter params (role.id=6fd3c8340d12478680af497dd225c736, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=48141af292e144caa3ea38613964403e) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.797724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.799125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467a5d0-7cca-429f-a142-e4b5d10a75e5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.800648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6a5b7-694b-4914-8038-74d984c5c50e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.815190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 498/1001] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/role_assignments?user.id=48141af292e144caa3ea38613964403e&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=6fd3c8340d12478680af497dd225c736 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:26.832027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 504/1002] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:27.272578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315bbb9-0cc3-4c0f-b93e-15e2545759fd None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.273123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315bbb9-0cc3-4c0f-b93e-15e2545759fd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.273633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315bbb9-0cc3-4c0f-b93e-15e2545759fd None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.275584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 499/1003] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:27.283542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.284284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.284284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.304443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:27.306102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:27.389571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.405499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 505/1004] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:27.429401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315bbb9-0cc3-4c0f-b93e-15e2545759fd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.476163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.580569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315bbb9-0cc3-4c0f-b93e-15e2545759fd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.627101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.628098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.628486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.628864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.629636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.631161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:27.644444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:27.646168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:27.647399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:27.649306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 500/1005] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:27.656218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.787221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0ad96-1256-4a9a-9cfe-896e0147866e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.824880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.825550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.825832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.826106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.826809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] RBAC: Adding query filter params (name=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:27.827142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.828533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.843420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 506/1006] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:27.847205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.984925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.013942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80a23a-72cd-494f-8859-16a8f22087c6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.078103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.079083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.079606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.080094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.081468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:28.082017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.084677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae7ef0-9c06-4a05-8da1-d962381e12dc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.099784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 501/1007] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:28.106192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.190555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.262565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a403-fbdd-4f78-93f0-bdae0f4a3fee None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.341649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cc39e-a89d-4a2d-99ee-ef57f515eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.347088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cc39e-a89d-4a2d-99ee-ef57f515eeed None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.347088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cc39e-a89d-4a2d-99ee-ef57f515eeed None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.347088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cc39e-a89d-4a2d-99ee-ef57f515eeed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.347088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cc39e-a89d-4a2d-99ee-ef57f515eeed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.347617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cc39e-a89d-4a2d-99ee-ef57f515eeed None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.403192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 507/1008] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:29.149041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9cd76-0a0e-42be-a442-93d98ec217be None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.149894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9cd76-0a0e-42be-a442-93d98ec217be None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.149894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9cd76-0a0e-42be-a442-93d98ec217be None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.151213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 502/1009] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:29.156814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57061b38-3c89-4fae-9132-a92f09855ae8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.157186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57061b38-3c89-4fae-9132-a92f09855ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.157435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57061b38-3c89-4fae-9132-a92f09855ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.176502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-57061b38-3c89-4fae-9132-a92f09855ae8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:29.177719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-57061b38-3c89-4fae-9132-a92f09855ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:29.268926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 508/1010] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:29 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:29.292714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9cd76-0a0e-42be-a442-93d98ec217be None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.489953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9cd76-0a0e-42be-a442-93d98ec217be None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.502844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232810-e5c7-4fe5-87a0-9f8646d4162d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.503705 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232810-e5c7-4fe5-87a0-9f8646d4162d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.503705 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232810-e5c7-4fe5-87a0-9f8646d4162d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.505882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 509/1011] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:29.513274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.513646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.514014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.534781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:29.536188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:29.558893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.560318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.560777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.561272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] PATH_INFO: `/v3/services/image` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.562477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.565306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.579590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:29.588825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:29.591837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:29.595090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 503/1012] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/services/image => generated 86 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:29.598556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.628571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.631681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 510/1013] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:29 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:29.673736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.782816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9285c-6e09-40bd-9534-9a1972787e39 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.827992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba15a-9074-4699-bc3f-fc37750fcd52 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.847076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.848252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.848748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.849225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.850541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] RBAC: Adding query filter params (name=image) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:29.851094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.853759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.863129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 504/1014] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:29.869204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.876860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.877666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.877948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.878228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.879495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.881177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.882848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 511/1015] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:29.889453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.941512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.021976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d390ea4-aa58-4858-8d70-2a381fe02c8e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.027693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.063251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6920a16-e3e1-4f1c-b77e-82ce347bed95 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.091871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eb05b-4991-44aa-8bac-5d5680809c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.093464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eb05b-4991-44aa-8bac-5d5680809c66 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.094239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eb05b-4991-44aa-8bac-5d5680809c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.094989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eb05b-4991-44aa-8bac-5d5680809c66 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.096735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6eb05b-4991-44aa-8bac-5d5680809c66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.099569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6eb05b-4991-44aa-8bac-5d5680809c66 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.126229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.127242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.127800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.128218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.129652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] RBAC: Adding query filter params (type=image) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:30.130384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.132885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26114cc-49f0-40e4-a154-c2d2f19f1fae None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.142021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 505/1016] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:30.159228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 512/1017] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:29 2022] POST /identity/v3/projects => generated 311 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:30.802692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c1fae-f7c4-49b1-894b-b92223123c36 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.803497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c1fae-f7c4-49b1-894b-b92223123c36 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.803497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c1fae-f7c4-49b1-894b-b92223123c36 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.804852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 506/1018] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:30.810489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.811348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.811348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.830770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:30.832090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:30.914412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 513/1019] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:30 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:30.938150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c1fae-f7c4-49b1-894b-b92223123c36 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.102761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c1fae-f7c4-49b1-894b-b92223123c36 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.144194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.145183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.145835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.146230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.159322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:31.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:31.169968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:31.172307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:31.174659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 507/1020] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:31.175513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.201878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.298658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd77d8-1b94-427b-a8d5-ac8cbb63e7d8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.338002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.338679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.338957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.339280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.340272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:31.340581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.341993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.359164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 514/1021] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:31.360399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.363483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.478783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9d3f3-44e2-4680-983b-3c91572da266 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.522959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.523799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.524108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.524357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.536309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.537784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:31.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:31.567505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:31.569708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:31.569708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:31.569708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:31.569708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:31.569708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 508/1022] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:31.573159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.678253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.708285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7991a8b-41bd-4300-9c27-391d17b000e3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.750235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.751026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.751580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.751984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.752893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:31.753328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.754918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.776097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 515/1023] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:31.778313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.848267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.908216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4d85d-3a4a-4893-a82d-d1484df4edcc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.953630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.954705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.955237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.955673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.967269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.969030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:31.985381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:31.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:31.998620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 509/1024] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:32.003180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.155240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3909b-1c11-4633-89dd-3640b2d9eeac None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.194599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.195311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.195628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.195877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.196698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:32.197005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.198472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a64b02-d022-4abc-a1e5-2121f527c017 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.218104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.223675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 516/1025] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:32.230843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.419039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7052-455f-45ce-9d0f-86a4ef69f109 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.447491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.448669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c52b75-e207-4418-8d47-936b4c6cfc8a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.448846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c52b75-e207-4418-8d47-936b4c6cfc8a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.449415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c52b75-e207-4418-8d47-936b4c6cfc8a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.451386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 517/1026] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:32.457145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.457421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.457724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.475726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:32.476930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:32.478105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.479488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.481442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.482330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.554624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 518/1027] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:32 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:32.580774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.609077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.610545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.737317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 510/1028] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:32.740268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2faabe-1fcd-4333-a493-efabb3302664 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.743006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.760109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.797267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.805809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.806702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.807120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.807440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.809332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.810879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.812909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 519/1029] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:32.818687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.862470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492c0d9-9689-478e-aae7-11e1f8a41138 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.911141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.911943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.912221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.912490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.926122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.927625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:32.947935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:32.950278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:32.952171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:32.952171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:32.952171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:32.952171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:32.952171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 511/1030] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:32.955636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.958127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02e627-caed-4101-9f48-eebd233c637c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.002337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0af63d-488c-4079-8eb0-09fbaba40cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.002986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0af63d-488c-4079-8eb0-09fbaba40cc1 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.003389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0af63d-488c-4079-8eb0-09fbaba40cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.004244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0af63d-488c-4079-8eb0-09fbaba40cc1 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.004586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0af63d-488c-4079-8eb0-09fbaba40cc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.006198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0af63d-488c-4079-8eb0-09fbaba40cc1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.032100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 520/1031] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:20:32 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:33.034412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949154b-f242-486c-b71f-3515ea1acddd None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.034908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949154b-f242-486c-b71f-3515ea1acddd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.035406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949154b-f242-486c-b71f-3515ea1acddd None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.037775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 521/1032] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:33.046407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.046849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.047319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.051957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:33.077811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:33.080220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:33.112053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0aa11-9ced-4b45-a75c-3fb95af75acd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.148937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:33.182224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.184100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.184729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.185229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.186268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:33.186873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 522/1033] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:33 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:33.187149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.188877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.204164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 512/1034] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:33.213927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.229367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.392985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cfaac-94c6-4ff9-bdc9-ede11e3f5c75 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.393445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae26666-bc0b-4c36-b111-a100cbf40055 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.429256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:33.442707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.443727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.444153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.444559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.445635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.447586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.454871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.456159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.456598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.457045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.468772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 513/1035] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:33 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:33.471915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.473783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:33.490305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:33.492084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:33.494114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 523/1036] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:33.499851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.700363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa2f6-1601-49a2-9496-3230f193dbf3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.754590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.755317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.755596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.755869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.756691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:33.757003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.758482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3047df3-3449-4d70-a8e0-fee144585527 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.774147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 514/1037] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:33.782119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.869911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:33.977717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9450c30-5d9c-436f-a56f-d1a6d79774db None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.044101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.045292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.045621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.046070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.047640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] RBAC: Adding query filter params (role.id=6fd3c8340d12478680af497dd225c736, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=054c95ae7aa645b7ade6390e9bf0059f) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:34.048172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.050709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34567ca-3a2c-4d9c-a29a-bae982fa99f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.067108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 524/1038] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:33 2022] GET /identity/v3/role_assignments?user.id=054c95ae7aa645b7ade6390e9bf0059f&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=6fd3c8340d12478680af497dd225c736 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:36.218218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28b64-2059-4e61-a225-909a8526ddfe None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.218786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28b64-2059-4e61-a225-909a8526ddfe None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.218786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28b64-2059-4e61-a225-909a8526ddfe None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.220320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 515/1039] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:36.226838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a0c1-f68f-48c0-a88b-45835242f037 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.227397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a0c1-f68f-48c0-a88b-45835242f037 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.227880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a0c1-f68f-48c0-a88b-45835242f037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.254315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9d7a0c1-f68f-48c0-a88b-45835242f037 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:36.255826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9d7a0c1-f68f-48c0-a88b-45835242f037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:36.339601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 525/1040] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:36.359008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f28b64-2059-4e61-a225-909a8526ddfe None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.446809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.493770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ae7a4-8d5c-430d-bfe7-63361face8a3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.494033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ae7a4-8d5c-430d-bfe7-63361face8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.494222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ae7a4-8d5c-430d-bfe7-63361face8a3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.496541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 526/1041] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:36.505201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.505716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.506222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.534564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:36.536709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:36.547363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f28b64-2059-4e61-a225-909a8526ddfe None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.573066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.590368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.591216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.591624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.591906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.603738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:36.612826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:36.613881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:36.614749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 516/1042] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:36.620010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.646958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 527/1043] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:36.680967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.769608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96521ec1-3def-44dc-aac0-e7a1481d0414 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.771653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.814405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.815325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.815804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.816155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.817142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:36.817490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.819332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.828326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d64db-a7f4-4ffd-aab6-8b097dd400aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.833385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 517/1044] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:36.841588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.872714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.874174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.874468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.874746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] PATH_INFO: `/v3/services/image` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.875605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.877699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:36.892020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:36.893480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:36.894357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 528/1045] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/services/image => generated 86 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:36.899480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.946874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.990377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf7484-2feb-4a39-b369-c73bcf889969 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.029631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c54d8-8677-4864-9565-3c60f09f87e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.030429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c54d8-8677-4864-9565-3c60f09f87e5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.030709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c54d8-8677-4864-9565-3c60f09f87e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.030984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c54d8-8677-4864-9565-3c60f09f87e5 None admin] PATH_INFO: `/v3/users/e8dbff5a0de64164902225ff50308ae3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.037393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c126b-35f3-4fc1-8509-af4efeeb58ba None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.050149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c54d8-8677-4864-9565-3c60f09f87e5 None admin] RBAC: Authorizing `identity:get_user(user_id=e8dbff5a0de64164902225ff50308ae3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.051698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c54d8-8677-4864-9565-3c60f09f87e5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.053504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 518/1046] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/users/e8dbff5a0de64164902225ff50308ae3 => generated 285 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:37.054795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f0f3f-9a11-44f6-8546-57ceb626a49b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.055100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f0f3f-9a11-44f6-8546-57ceb626a49b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.055382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f0f3f-9a11-44f6-8546-57ceb626a49b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.056908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 519/1047] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:37.060043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f0f3f-9a11-44f6-8546-57ceb626a49b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.085417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.087845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.089666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.090181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.090686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.092000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] RBAC: Adding query filter params (name=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.094169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.094169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6db66e-3622-4c45-a28c-5587067aed78 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.103510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 529/1048] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:37.106251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.106780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.107328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.133031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:37.134239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:37.181453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f0f3f-9a11-44f6-8546-57ceb626a49b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.221936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.222652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.222876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.223184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.226129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 530/1049] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:37.227021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.261966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.306558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.308032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.335111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.349610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe22bd2-391e-4968-8683-15614e7b54c9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.407072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.407801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.408086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.408355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.409111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] RBAC: Adding query filter params (type=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.409413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.410880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.417455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 531/1050] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.418701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.440755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 520/1051] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:37.442241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.555330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.566856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd710cf-c904-422b-9e49-8ef818275b5c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.582073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf1255-17e5-460e-8ddc-0e35dcb2cc59 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.607948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.609337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.609752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.610071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.612264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.625165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:37.631893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.632994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.633505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:37.633642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:37.636788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:37.639175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:37.640600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.640766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 532/1052] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:37.640925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] RBAC: Adding query filter params (interface=public, service_id=d34188c192f048babd3deae28729a31b, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.641071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.641221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.641221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.649362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 521/1053] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/endpoints?service_id=d34188c192f048babd3deae28729a31b&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.650627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.766183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49664c42-dcda-4c86-b62a-8d13467b4890 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.772909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26834c-2125-49ed-a6dc-f04d46c8e2ac None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.805941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.807178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.807576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.808499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.810119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] RBAC: Authorizing `identity:get_project(project_id=9a4187e03183485a8f183b053b9f11f3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.811775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.811919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.812465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.812752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.813029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.813787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.814110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.814348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 533/1054] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.815353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.815524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.829983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 522/1055] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.830749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.841957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.894353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.941249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04f3d-feea-47aa-8ec0-39de8dfcba1c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.953271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ec01-92e8-47ca-bd01-44b8ed36c1ac None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.983315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.983988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.984274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.984549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.985244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.986640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.993629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 534/1056] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:20:37.993844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.994442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.994736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.995805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.996222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.997235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=9a4187e03183485a8f183b053b9f11f3, user.id=e8dbff5a0de64164902225ff50308ae3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.997686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.999517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.007443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 523/1057] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/role_assignments?user.id=e8dbff5a0de64164902225ff50308ae3&scope.project.id=9a4187e03183485a8f183b053b9f11f3&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:38.149346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.194639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b601c-40bd-421a-9f0e-90359ff91296 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.262099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.263561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.264066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.264571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.285222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.287804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:38.320662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:38.321707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:38.322643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:38.322643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:38.322643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:38.322643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:38.323290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 535/1058] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:38.332008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.534482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599494-3635-4ad4-b10c-39416c7805ac None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.564411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.614029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.615250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.615735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.616217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.617622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:38.618182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.621435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.650869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 524/1059] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:38.652483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.787722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc645a-b913-4b4b-a3e9-e756e498c025 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.828676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.829489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.829774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.830059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.832420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.842002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.843530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:38.859988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:38.861126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:38.862090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 536/1060] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:38.871882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:39.078499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94ffa7-a808-4938-b555-07e69fa30a9e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:39.144383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:39.145776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:39.146300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:39.146793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:39.148418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:39.148973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:39.152728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:39.188140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 525/1061] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:39.197357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:39.392924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498345c-38bd-4872-90e6-a33271bbbda0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:39.460344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:39.461693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:39.462190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:39.462683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:39.483103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:39.485808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:39.518547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:39.520623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:39.529223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:39.529223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:39.529223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:39.529223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:39.529223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 537/1062] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:39.533670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:39.591832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:39.739437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d80d-c87c-43fd-b1bc-33e15bdfa211 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:39.809709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:39.810888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:39.811447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:39.811957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:39.813379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:39.813903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:39.816433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:39.847022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 526/1063] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:39.849148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:39.939917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.044667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51ac1e-abec-466d-99db-5d0a376998d8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.112247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.113707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.114212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.114707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.128998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.131812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8e12dba7-26c9-4639-89fb-99b79cf9f7e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:40.161537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:40.163768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:40.169688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 538/1064] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:40.177325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.249358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29ef9d-c4c6-45eb-9849-c61f2761d342 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.249358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29ef9d-c4c6-45eb-9849-c61f2761d342 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.249773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29ef9d-c4c6-45eb-9849-c61f2761d342 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.255660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 539/1065] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:40.261510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.261999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.262156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.283942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:40.285928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:40.346782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.367590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d984c5-badd-4603-972b-ae09f9a14989 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.393188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 540/1066] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:40.431307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.436151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.438250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.438708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.439183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.440574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:40.441082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.443651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.475098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 527/1067] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:40.478461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.489421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.607514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b315a30-1620-4f3f-b304-e908744eb407 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.658201 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db96f8f-30b6-4eba-9478-cecc7c6676f4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.671449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961e89d-41f1-425a-a317-383c73f5edbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.672588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961e89d-41f1-425a-a317-383c73f5edbc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.673052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961e89d-41f1-425a-a317-383c73f5edbc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.673511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961e89d-41f1-425a-a317-383c73f5edbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.691572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961e89d-41f1-425a-a317-383c73f5edbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a961e89d-41f1-425a-a317-383c73f5edbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:40.705624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:40.706805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:40.707835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:40.708933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 541/1068] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:40.710232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f6f6e-51c6-4d55-93c3-b228247f732d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.710612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f6f6e-51c6-4d55-93c3-b228247f732d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.710939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f6f6e-51c6-4d55-93c3-b228247f732d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.712408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 542/1069] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:40.714970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f6f6e-51c6-4d55-93c3-b228247f732d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.715771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbc2ca-9d40-4507-ae2f-809097ecb7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.717227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbc2ca-9d40-4507-ae2f-809097ecb7cd None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.717676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbc2ca-9d40-4507-ae2f-809097ecb7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.718123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbc2ca-9d40-4507-ae2f-809097ecb7cd None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/054c95ae7aa645b7ade6390e9bf0059f/roles/6fd3c8340d12478680af497dd225c736` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.721328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbc2ca-9d40-4507-ae2f-809097ecb7cd None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=054c95ae7aa645b7ade6390e9bf0059f, role_id=6fd3c8340d12478680af497dd225c736)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.723876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbc2ca-9d40-4507-ae2f-809097ecb7cd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.743861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 528/1070] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:40 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/054c95ae7aa645b7ade6390e9bf0059f/roles/6fd3c8340d12478680af497dd225c736 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:20:40.746204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.746546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.746900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.766021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:40.767381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:40.772815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.859792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.865136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 529/1071] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:40.870857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f6f6e-51c6-4d55-93c3-b228247f732d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.895357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.917897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.919100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.919646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.920189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.921432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:40.921884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.923567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.940056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 543/1072] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:40.946358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.051165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950d440-f717-4ff5-a987-76946c0ce45e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.110696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.111807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.112137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.112479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] PATH_INFO: `/v3/services/image` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.114042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.115644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.132049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9614e-1b17-4521-8982-a700b16b57ab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:41.136353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:41.138233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:20:41.141748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 530/1073] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/services/image => generated 86 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:41.145657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.153627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.201146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.203001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.203752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.204336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] PATH_INFO: `/v3/users/e8dbff5a0de64164902225ff50308ae3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.206611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] RBAC: Authorizing `identity:get_user(user_id=e8dbff5a0de64164902225ff50308ae3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.208429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.210949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 544/1074] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/users/e8dbff5a0de64164902225ff50308ae3 => generated 285 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:41.220800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.268453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.278558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cbd13-96fd-41a5-9d6b-65334f28c6ce None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.333582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.334808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.335347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.335667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.336670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] RBAC: Adding query filter params (name=image) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:41.336980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.338705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.350594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 531/1075] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:41.351463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.383622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c2c74-86f7-4b51-8494-ca293a8159fe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.435456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.436628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.436913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.437194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.439036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] RBAC: Authorizing `identity:get_project(project_id=9a4187e03183485a8f183b053b9f11f3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.441028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.449468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 545/1076] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3 => generated 311 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:41.451303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.451468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.489481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1850-2d0d-4163-aa18-8245bdb5af2a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.547993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.549166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.549685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.550225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.551564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] RBAC: Adding query filter params (type=image) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:41.552125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.554769 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.567098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 532/1077] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:41.569901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.583341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.596979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a6b5-cfba-4eed-b28b-19cc7f71054e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.643963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1246a-baa1-4891-affc-ed154cedad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.645548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1246a-baa1-4891-affc-ed154cedad3f None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.646017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1246a-baa1-4891-affc-ed154cedad3f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.646469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1246a-baa1-4891-affc-ed154cedad3f None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3/users/e8dbff5a0de64164902225ff50308ae3/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.649501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1246a-baa1-4891-affc-ed154cedad3f None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4187e03183485a8f183b053b9f11f3, user_id=e8dbff5a0de64164902225ff50308ae3, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.652597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1246a-baa1-4891-affc-ed154cedad3f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.674973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 546/1078] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:41 2022] PUT /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3/users/e8dbff5a0de64164902225ff50308ae3/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:41.728070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63e1f-de73-4e3a-9936-f5eda832b230 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.771467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.780573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df61aa-ae3b-48c3-a8ac-a445d72f5ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.780950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df61aa-ae3b-48c3-a8ac-a445d72f5ad9 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.781198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df61aa-ae3b-48c3-a8ac-a445d72f5ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.781478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df61aa-ae3b-48c3-a8ac-a445d72f5ad9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.782286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df61aa-ae3b-48c3-a8ac-a445d72f5ad9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.783797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df61aa-ae3b-48c3-a8ac-a445d72f5ad9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.811843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 533/1079] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:42.934735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc87d0e-666a-415f-9aad-150ddc3eec0a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.935270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc87d0e-666a-415f-9aad-150ddc3eec0a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.935491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc87d0e-666a-415f-9aad-150ddc3eec0a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.936860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 547/1080] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:42.942815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.943658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.943658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.961054 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:42.962257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:43.060465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 534/1081] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:43.086292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc87d0e-666a-415f-9aad-150ddc3eec0a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.270903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc87d0e-666a-415f-9aad-150ddc3eec0a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.340384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.341820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.342328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.342823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.364108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:43.381724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:43.384030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:43.386213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 548/1082] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:43.391934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.530186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b907-4c9f-467e-acce-df5c1e5e6d47 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.547048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:43.574284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.575197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.575606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.576060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.576969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:43.577399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.578927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.589534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 535/1083] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:43.595742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.614503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:43.745762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd65a29-e1a3-44e6-afd9-0ced449c9288 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.813072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.814351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.814836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.815381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.835310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.837907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2a9eb781-82f3-4954-95e9-3364321b5ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:43.868397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:43.870636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:43.880423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:43.880423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:43.880423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:43.880423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:43.880423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 549/1084] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:43.883762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.985700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9600-cfb3-419b-9f77-0235b13db90a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.986509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9600-cfb3-419b-9f77-0235b13db90a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.986509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9600-cfb3-419b-9f77-0235b13db90a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.987942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 550/1085] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:43.993472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb42b16-f5e2-4362-99f8-1ba322391512 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.993768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb42b16-f5e2-4362-99f8-1ba322391512 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.994083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb42b16-f5e2-4362-99f8-1ba322391512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.020998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fdb42b16-f5e2-4362-99f8-1ba322391512 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:44.023515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fdb42b16-f5e2-4362-99f8-1ba322391512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:44.087121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9509f-2d8e-4676-8ced-f671662b3c01 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.129576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.153999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 551/1086] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:44.154198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.154297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d0779-7222-46e7-875f-5a45c43349ac None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.154370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.154440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d0779-7222-46e7-875f-5a45c43349ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.154541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.154614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d0779-7222-46e7-875f-5a45c43349ac None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.154692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.154692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:44.154692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.154880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 552/1087] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:44.154950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.156834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.157562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.157809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.180185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 536/1088] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:44.180185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.187455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:44.189577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:44.235756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.300003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 553/1089] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:44.301097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.343920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44577c83-9fc3-4a82-a26f-684cbc879115 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.391945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.392802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.393192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.393530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.422176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc737e5-f5c9-450e-815f-25170e722b27 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.461136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.476729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.477775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.478216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.478657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.481371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.483374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.488994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.490659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:44.507430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:44.509372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 554/1090] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:44.511775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.531612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.633324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403505-10e3-48a1-bb97-984922ef368e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.644776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 537/1091] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:44.646374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.675193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.676187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.676599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.676998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.691399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:44.703408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:44.704507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:44.705702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 555/1092] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:44.707121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.766302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.785875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70f0ad-4672-458e-aa1b-ac4bf2d24971 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.824724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.825397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.825680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.825960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.826801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:44.827155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.828609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.833169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.836110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b34f84-688a-45aa-8d4b-87462030fbce None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.855367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 538/1093] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:44.856233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.876560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.877374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.877661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.877941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.891032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.892483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:44.912428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:44.913564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:44.914474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:44.914474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:44.914474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:44.914474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:44.914474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 556/1094] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:44.914474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.012893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba505fea-ed09-4127-80a0-f63d31c3674b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.043446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.049714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8eacb-b216-4286-bbc7-1bb3969a4539 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.063441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.064243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.064539 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.064821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.065607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:45.065922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.067413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.077921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 539/1095] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:45.078606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.114431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.115431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.115790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.116201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.157786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.205474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.207128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.211877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280da8-34e9-4895-8c42-29954f14cc80 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.259134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.259944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.260321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.260703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.261667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:45.262028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.263670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.278713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 540/1096] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:45.279792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.348462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.356657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 557/1097] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:45.365094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.467821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543b5ad-08bd-4614-ac69-7594131bde6f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.491990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.529178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.530007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.530477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.530953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] PATH_INFO: `/v3/users/e8dbff5a0de64164902225ff50308ae3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.532782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] RBAC: Authorizing `identity:get_user(user_id=e8dbff5a0de64164902225ff50308ae3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.535008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.539018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 541/1098] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:45 2022] GET /identity/v3/users/e8dbff5a0de64164902225ff50308ae3 => generated 285 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:45.545250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.550912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6b0bb-cd2b-4e8e-a529-15422e17f4fb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.609451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.610653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.611201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.611606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.632648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.636509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:45.666202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:45.668023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:45.669714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:45.669714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:45.669714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:45.669714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:45.670709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 558/1099] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:45.675932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.724300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ef5d6-e2d1-45d6-9ccd-c28d7caab58e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.750984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.779091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.779927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.780339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.780793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.830050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46293-c68f-4b19-9dbd-502d8dd7aea1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.856716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.881116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.882737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.888918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.889766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.890183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.890582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.891519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:45.891952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.893523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.907978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 559/1100] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:45.914258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.016995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 542/1101] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:46.024105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.054686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20a2f-5a5f-486a-81b2-e90a2b4117b5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.067599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:46.097472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.098407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.098784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.099214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.108026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.110627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:46.132264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:46.134135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:20:46.136478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 560/1102] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:46.141543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.142269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:46.165948 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346327e-16d6-4b4d-afd3-b3aae78a3e68 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.223578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.224553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.224942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.225317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.226901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] RBAC: Authorizing `identity:get_project(project_id=9a4187e03183485a8f183b053b9f11f3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.228633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.231527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 543/1103] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3 => generated 311 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:46.239018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.265448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41fb3f-aba7-41c1-9b2c-a9dd097ec334 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.302828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.303428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.303711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.303982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.304766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:46.305075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.306517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.318774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 561/1104] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:46.325222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.358767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:46.377853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9422b-29d3-4c84-a5fe-d718aefb8844 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.419982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.420768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.421050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.421329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.422187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=9a4187e03183485a8f183b053b9f11f3, user.id=e8dbff5a0de64164902225ff50308ae3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:46.422502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.424014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617c2a-a046-4013-9f31-873b08728e9d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.430690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 544/1105] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/role_assignments?user.id=e8dbff5a0de64164902225ff50308ae3&scope.project.id=9a4187e03183485a8f183b053b9f11f3&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:46.437378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07663c5-95f6-47e0-8e11-b88aca9e7584 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.478926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.480179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.480457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.480728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.481589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] RBAC: Adding query filter params (role.id=6fd3c8340d12478680af497dd225c736, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=054c95ae7aa645b7ade6390e9bf0059f) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:46.481899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.483440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e884-47c6-406f-aad5-64fc4ec075fa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.490696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 562/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/role_assignments?user.id=054c95ae7aa645b7ade6390e9bf0059f&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=6fd3c8340d12478680af497dd225c736 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:48.597327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d52cbd-2b1f-4995-84ae-687ff03c9ec4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.598198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d52cbd-2b1f-4995-84ae-687ff03c9ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.598198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d52cbd-2b1f-4995-84ae-687ff03c9ec4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.599275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 545/1107] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:48.605067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.605595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.605595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.624265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:48.625502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:48.699957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 563/1108] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:48.725638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d52cbd-2b1f-4995-84ae-687ff03c9ec4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.745301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.845633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d52cbd-2b1f-4995-84ae-687ff03c9ec4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.887936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8a070-e12e-4696-97b4-017a0ed4bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.888668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a070-e12e-4696-97b4-017a0ed4bf75 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.888949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a070-e12e-4696-97b4-017a0ed4bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.889221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a070-e12e-4696-97b4-017a0ed4bf75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.890522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8a070-e12e-4696-97b4-017a0ed4bf75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.892056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8a070-e12e-4696-97b4-017a0ed4bf75 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.893835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 546/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.900419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.944545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:49.072131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcace1b-df3a-4398-a0ed-0048c0ce6c27 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:49.131234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef6e7-ac71-4908-a6af-add3a9bd5875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:49.132361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef6e7-ac71-4908-a6af-add3a9bd5875 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:49.132871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef6e7-ac71-4908-a6af-add3a9bd5875 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:49.133377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef6e7-ac71-4908-a6af-add3a9bd5875 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:49.134736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef6e7-ac71-4908-a6af-add3a9bd5875 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:49.138548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef6e7-ac71-4908-a6af-add3a9bd5875 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.166341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 564/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:20:48 2022] POST /identity/v3/users => generated 286 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:51.352685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a46979-1034-4f6e-a1d6-c77848488340 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.353509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a46979-1034-4f6e-a1d6-c77848488340 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.353666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a46979-1034-4f6e-a1d6-c77848488340 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.355132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 547/1111] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:51.361603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.362150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.362150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.381135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:51.382347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:51.458819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 565/1112] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:51.482969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a46979-1034-4f6e-a1d6-c77848488340 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.578024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.607433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a46979-1034-4f6e-a1d6-c77848488340 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:51.646973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:51.647924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.648246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.648564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] PATH_INFO: `/v3/roles/852d973419ee442c80da4214c3df867b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.650144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] RBAC: Authorizing `identity:get_role(role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:51.652737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:51.654239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 548/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/roles/852d973419ee442c80da4214c3df867b => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:51.662098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.674697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.786200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4467bc-8df9-42cd-b576-8452deaead67 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:51.822020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:51.823164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.823549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.823832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] PATH_INFO: `/v3/users/439bf1e6d59b4ca69193236e1ce650ba` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.837848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] RBAC: Authorizing `identity:get_user(user_id=439bf1e6d59b4ca69193236e1ce650ba)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:51.839757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:51.841496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 566/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/users/439bf1e6d59b4ca69193236e1ce650ba => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:51.847579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.965553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bda80-0af4-481d-80b8-5b27e7387706 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.006199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.006952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.007294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.007608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.034974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.088916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.090967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.203422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 549/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:52.209569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.284165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.320949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf4ac-6b59-465e-bc4d-f8ad5b673074 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.359867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087838d-5bf5-4535-8a74-2bed7d355ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.360738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087838d-5bf5-4535-8a74-2bed7d355ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.361020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087838d-5bf5-4535-8a74-2bed7d355ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.361298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087838d-5bf5-4535-8a74-2bed7d355ab0 None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.362499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087838d-5bf5-4535-8a74-2bed7d355ab0 None admin] RBAC: Authorizing `identity:get_project(project_id=9a4187e03183485a8f183b053b9f11f3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.363986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087838d-5bf5-4535-8a74-2bed7d355ab0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.365713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 567/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:52.373138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.433616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.487630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652b776-694d-48ea-8651-b14dbf990d50 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.534907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.535510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.535797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.536075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.536922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] RBAC: Adding query filter params (role.id=852d973419ee442c80da4214c3df867b, scope.project.id=9a4187e03183485a8f183b053b9f11f3, user.id=439bf1e6d59b4ca69193236e1ce650ba) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:52.537239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.538702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fdf4a-f0ae-45e5-a300-c96538a9c335 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.544981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 550/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/role_assignments?user.id=439bf1e6d59b4ca69193236e1ce650ba&scope.project.id=9a4187e03183485a8f183b053b9f11f3&role.id=852d973419ee442c80da4214c3df867b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:54.736258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd497e-cbc3-41bb-815d-baff6ba331ce None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.737122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd497e-cbc3-41bb-815d-baff6ba331ce None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.737122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd497e-cbc3-41bb-815d-baff6ba331ce None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.738212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 568/1118] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:54.745315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.745986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.745986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.763312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:54.764594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:54.841309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 551/1119] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:54.865189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd497e-cbc3-41bb-815d-baff6ba331ce None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:54.981376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd497e-cbc3-41bb-815d-baff6ba331ce None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.017459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.018390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.018724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.019084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] PATH_INFO: `/v3/roles/852d973419ee442c80da4214c3df867b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.020928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] RBAC: Authorizing `identity:get_role(role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.022379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.023923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 569/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:54 2022] GET /identity/v3/roles/852d973419ee442c80da4214c3df867b => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:55.030736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:55.081563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:55.140477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1146922-f9cd-45a0-812b-533334182c0c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.178033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.179534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.179534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.179867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] PATH_INFO: `/v3/users/439bf1e6d59b4ca69193236e1ce650ba` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.180979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] RBAC: Authorizing `identity:get_user(user_id=439bf1e6d59b4ca69193236e1ce650ba)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.183106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.184888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 552/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:55 2022] GET /identity/v3/users/439bf1e6d59b4ca69193236e1ce650ba => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:55.191030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:55.195304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:55.297087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dac88f-9454-401c-9fa9-f702de9db5f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.333572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75331b15-720f-46e7-a2fa-263e36825743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.334446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75331b15-720f-46e7-a2fa-263e36825743 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.334730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75331b15-720f-46e7-a2fa-263e36825743 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.335004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75331b15-720f-46e7-a2fa-263e36825743 None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.336247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75331b15-720f-46e7-a2fa-263e36825743 None admin] RBAC: Authorizing `identity:get_project(project_id=9a4187e03183485a8f183b053b9f11f3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.337724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75331b15-720f-46e7-a2fa-263e36825743 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.340245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 570/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:20:55 2022] GET /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:55.346467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:55.456585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8b8f-1bec-4c8b-b84c-18f657d1d26c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.472577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:55.498365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247e58c-910c-4865-a8b1-20dc94d41ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.499311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e58c-910c-4865-a8b1-20dc94d41ef0 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.499598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e58c-910c-4865-a8b1-20dc94d41ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.499878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e58c-910c-4865-a8b1-20dc94d41ef0 None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3/users/439bf1e6d59b4ca69193236e1ce650ba/roles/852d973419ee442c80da4214c3df867b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.502290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247e58c-910c-4865-a8b1-20dc94d41ef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4187e03183485a8f183b053b9f11f3, user_id=439bf1e6d59b4ca69193236e1ce650ba, role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.503782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247e58c-910c-4865-a8b1-20dc94d41ef0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.518886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 553/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:55 2022] PUT /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3/users/439bf1e6d59b4ca69193236e1ce650ba/roles/852d973419ee442c80da4214c3df867b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:57.650174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542a885-97cf-4cb7-8109-44f1a67f8092 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.651389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542a885-97cf-4cb7-8109-44f1a67f8092 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.651389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542a885-97cf-4cb7-8109-44f1a67f8092 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.652733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 571/1124] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:57.661017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.661493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.661919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.683617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:57.684896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:57.766709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 554/1125] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:57.792532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542a885-97cf-4cb7-8109-44f1a67f8092 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.875485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:57.912008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542a885-97cf-4cb7-8109-44f1a67f8092 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.950702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.951716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.952067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.952391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] PATH_INFO: `/v3/roles/852d973419ee442c80da4214c3df867b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.953964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] RBAC: Authorizing `identity:get_role(role_id=852d973419ee442c80da4214c3df867b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.955434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.956892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 572/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:57 2022] GET /identity/v3/roles/852d973419ee442c80da4214c3df867b => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:57.963810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.071528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9983d8-bb10-4169-8bde-1ca6579a1154 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.074886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.110477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.111532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.111825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.112100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] PATH_INFO: `/v3/users/439bf1e6d59b4ca69193236e1ce650ba` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.113502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] RBAC: Authorizing `identity:get_user(user_id=439bf1e6d59b4ca69193236e1ce650ba)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.114952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.116684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 555/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:20:57 2022] GET /identity/v3/users/439bf1e6d59b4ca69193236e1ce650ba => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:58.126843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.237483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a6f0c-e9d5-477d-9e58-a123edb0d19d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.275029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.276012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.276478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.276757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.289107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.351533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.353039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.481823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 573/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:58.488171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.598084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5358b7-6cf2-4f2e-a4a9-a86063c4ca99 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.637077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334d2e5-1ec8-41e7-a5fa-3767f39fe733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.638051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334d2e5-1ec8-41e7-a5fa-3767f39fe733 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.638340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334d2e5-1ec8-41e7-a5fa-3767f39fe733 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.638665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334d2e5-1ec8-41e7-a5fa-3767f39fe733 None admin] PATH_INFO: `/v3/projects/9a4187e03183485a8f183b053b9f11f3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.639992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334d2e5-1ec8-41e7-a5fa-3767f39fe733 None admin] RBAC: Authorizing `identity:get_project(project_id=9a4187e03183485a8f183b053b9f11f3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.641487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334d2e5-1ec8-41e7-a5fa-3767f39fe733 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.643316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 556/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/projects/9a4187e03183485a8f183b053b9f11f3 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:58.650086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.697480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.764980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fced5a0-7ce4-421c-b2dc-e077a120b37b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.801578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.802243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.802522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.802796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.803674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] RBAC: Adding query filter params (role.id=852d973419ee442c80da4214c3df867b, scope.project.id=9a4187e03183485a8f183b053b9f11f3, user.id=439bf1e6d59b4ca69193236e1ce650ba) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:58.803985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.805442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0161f-8247-440f-bf8e-7a28297f6c5d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.812929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 574/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/role_assignments?user.id=439bf1e6d59b4ca69193236e1ce650ba&scope.project.id=9a4187e03183485a8f183b053b9f11f3&role.id=852d973419ee442c80da4214c3df867b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:00.980220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180870b7-4d7b-4bf0-9aa6-ee59ec1246a0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.981022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180870b7-4d7b-4bf0-9aa6-ee59ec1246a0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.981022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180870b7-4d7b-4bf0-9aa6-ee59ec1246a0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.982553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 557/1131] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:00.988656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.989216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.989216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.007416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:01.008670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:01.079816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 575/1132] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:01.104262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180870b7-4d7b-4bf0-9aa6-ee59ec1246a0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:01.149219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:01.207926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180870b7-4d7b-4bf0-9aa6-ee59ec1246a0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:01.243117 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70179dd-5c09-4f34-8ba6-e34235abcf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:01.244368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70179dd-5c09-4f34-8ba6-e34235abcf6d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.244651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70179dd-5c09-4f34-8ba6-e34235abcf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.244930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70179dd-5c09-4f34-8ba6-e34235abcf6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.246136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70179dd-5c09-4f34-8ba6-e34235abcf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:01.247623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70179dd-5c09-4f34-8ba6-e34235abcf6d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:01.249284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 558/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:01.255208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:01.288811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:01.359871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b097e0-23b1-49f6-b747-51786f84fa42 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:01.396662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96521051-3226-407a-848f-367eca10d16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:01.397317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96521051-3226-407a-848f-367eca10d16b None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.397600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96521051-3226-407a-848f-367eca10d16b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.397880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96521051-3226-407a-848f-367eca10d16b None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.398603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96521051-3226-407a-848f-367eca10d16b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:01.400304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96521051-3226-407a-848f-367eca10d16b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:01.429000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 576/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:21:01 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:03.581940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90ee08-f5e9-4fcf-b828-905a3ed39117 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.582569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90ee08-f5e9-4fcf-b828-905a3ed39117 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.582569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90ee08-f5e9-4fcf-b828-905a3ed39117 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.583895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 559/1135] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:03.590217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.590762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.590762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.615153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:03.617215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:03.714728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 577/1136] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:03.742697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90ee08-f5e9-4fcf-b828-905a3ed39117 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.866461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90ee08-f5e9-4fcf-b828-905a3ed39117 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.877961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:03.912266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b012b-7aba-4d0a-87c2-597b11548b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.913076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b012b-7aba-4d0a-87c2-597b11548b29 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.913358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b012b-7aba-4d0a-87c2-597b11548b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.913630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b012b-7aba-4d0a-87c2-597b11548b29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.914855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b012b-7aba-4d0a-87c2-597b11548b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.916424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b012b-7aba-4d0a-87c2-597b11548b29 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.918130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 560/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:03.924210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.943377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:04.030610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9c892-7266-4dcd-9485-09a44ef6ff34 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:04.066249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b205d-31f4-40ed-8594-caf824b31337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:04.066934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b205d-31f4-40ed-8594-caf824b31337 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:04.067258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b205d-31f4-40ed-8594-caf824b31337 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:04.067540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b205d-31f4-40ed-8594-caf824b31337 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:04.068260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b205d-31f4-40ed-8594-caf824b31337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:04.069720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b205d-31f4-40ed-8594-caf824b31337 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:04.091893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 578/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:21:03 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:21:06.305363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a614ad1-7c45-432d-bfcd-93e0f1684835 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.306442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a614ad1-7c45-432d-bfcd-93e0f1684835 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.306442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a614ad1-7c45-432d-bfcd-93e0f1684835 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.308017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 561/1139] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:06.314098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169238-32e5-4acb-a909-88856de5d096 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.314658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169238-32e5-4acb-a909-88856de5d096 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.314658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169238-32e5-4acb-a909-88856de5d096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.333198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-37169238-32e5-4acb-a909-88856de5d096 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:06.334441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37169238-32e5-4acb-a909-88856de5d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:06.415717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 579/1140] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:06.440105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a614ad1-7c45-432d-bfcd-93e0f1684835 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.549207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a614ad1-7c45-432d-bfcd-93e0f1684835 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.586064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175adbe-a523-4470-991e-084a30502123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.586763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175adbe-a523-4470-991e-084a30502123 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.587422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175adbe-a523-4470-991e-084a30502123 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.587705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175adbe-a523-4470-991e-084a30502123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.598969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175adbe-a523-4470-991e-084a30502123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d175adbe-a523-4470-991e-084a30502123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:06.607347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:06.608598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:21:06.609662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 562/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:06.614803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37169238-32e5-4acb-a909-88856de5d096 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.617108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.724972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37169238-32e5-4acb-a909-88856de5d096 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.764419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.765093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.765376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.765650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.766403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:06.766718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.768162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.777961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 580/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:06.784032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175adbe-a523-4470-991e-084a30502123 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.820791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.904382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175adbe-a523-4470-991e-084a30502123 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.949851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.950795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.951271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.951726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] PATH_INFO: `/v3/users/eafb76ad294443a5ab98e1fc63874da7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.973561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] RBAC: Authorizing `identity:get_user(user_id=eafb76ad294443a5ab98e1fc63874da7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.975019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.976784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 563/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/users/eafb76ad294443a5ab98e1fc63874da7 => generated 286 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:06.982767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.050895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.114592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3320d3-ad20-45bd-aecc-c98fef833a3c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.151850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.152508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.152797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.153075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.223862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.225266 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.295577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.339784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 581/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:07.346044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.447049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.454787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f766-4ebf-4db5-800d-3522eeb542c0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.491097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3c8c1-a480-40e4-8e8f-dc82aeb15d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.491927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c8c1-a480-40e4-8e8f-dc82aeb15d1b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.492214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c8c1-a480-40e4-8e8f-dc82aeb15d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.492501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c8c1-a480-40e4-8e8f-dc82aeb15d1b None admin] PATH_INFO: `/v3/projects/e6ca0d2625fb488e8634d7535a031ad4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.494417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3c8c1-a480-40e4-8e8f-dc82aeb15d1b None admin] RBAC: Authorizing `identity:get_project(project_id=e6ca0d2625fb488e8634d7535a031ad4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.495941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3c8c1-a480-40e4-8e8f-dc82aeb15d1b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.497832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 564/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/projects/e6ca0d2625fb488e8634d7535a031ad4 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:07.505146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.621914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476934b-8991-4396-90b5-a19a3cc3fd83 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.661220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.661516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.661801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.662075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.662922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e6ca0d2625fb488e8634d7535a031ad4, user.id=eafb76ad294443a5ab98e1fc63874da7) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:07.663268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.664771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7ab4-847a-4882-bd8e-f0056630a826 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.671711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 582/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/role_assignments?user.id=eafb76ad294443a5ab98e1fc63874da7&scope.project.id=e6ca0d2625fb488e8634d7535a031ad4&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:09.761037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8c656-1348-4b18-8af7-77b182b434bd None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:09.761618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8c656-1348-4b18-8af7-77b182b434bd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:09.761618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8c656-1348-4b18-8af7-77b182b434bd None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:09.762958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 565/1147] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:21:09.768555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:09.769306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:09.769400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:09.786914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:09.788173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:09.812751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:09.864319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 583/1148] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:09.888269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8c656-1348-4b18-8af7-77b182b434bd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:09.994741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8c656-1348-4b18-8af7-77b182b434bd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.030288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.031105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.031392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.031677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.042895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:10.050645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:10.051765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:21:10.053556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 566/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:10.058171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.169279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efcd8-5e9b-45cd-8ee6-dd919b52e03d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.205276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.205959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.206245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.206519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.207305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:10.207620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.209011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.218038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 584/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:10.224657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.251171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:10.346014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a3a4-b1d2-4afb-b8a4-1d38d5dbf933 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.383869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.385363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.385647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.385922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] PATH_INFO: `/v3/users/eafb76ad294443a5ab98e1fc63874da7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.387177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] RBAC: Authorizing `identity:get_user(user_id=eafb76ad294443a5ab98e1fc63874da7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.388627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.390294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 567/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:10 2022] GET /identity/v3/users/eafb76ad294443a5ab98e1fc63874da7 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:10.397134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.436198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:10.568074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843998-f3e8-444d-8568-1e0feaa2a23a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.608148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0823135-e188-4ed6-893b-f9fe81261fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.609096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0823135-e188-4ed6-893b-f9fe81261fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.609379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0823135-e188-4ed6-893b-f9fe81261fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.609659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0823135-e188-4ed6-893b-f9fe81261fd6 None admin] PATH_INFO: `/v3/projects/e6ca0d2625fb488e8634d7535a031ad4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.611015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0823135-e188-4ed6-893b-f9fe81261fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=e6ca0d2625fb488e8634d7535a031ad4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.612557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0823135-e188-4ed6-893b-f9fe81261fd6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.614493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 585/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:10 2022] GET /identity/v3/projects/e6ca0d2625fb488e8634d7535a031ad4 => generated 311 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:10.622089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.734305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:10.745571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72114fbf-1768-4fe2-9eb0-60e44e56d4fc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.782573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d6569-b5c2-4e72-9e04-660b4cd93bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.783506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6569-b5c2-4e72-9e04-660b4cd93bbc None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.783817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6569-b5c2-4e72-9e04-660b4cd93bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.784095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6569-b5c2-4e72-9e04-660b4cd93bbc None admin] PATH_INFO: `/v3/projects/e6ca0d2625fb488e8634d7535a031ad4/users/eafb76ad294443a5ab98e1fc63874da7/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.786161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d6569-b5c2-4e72-9e04-660b4cd93bbc None admin] RBAC: Authorizing `identity:create_grant(project_id=e6ca0d2625fb488e8634d7535a031ad4, user_id=eafb76ad294443a5ab98e1fc63874da7, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.787661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d6569-b5c2-4e72-9e04-660b4cd93bbc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.807433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 568/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:21:10 2022] PUT /identity/v3/projects/e6ca0d2625fb488e8634d7535a031ad4/users/eafb76ad294443a5ab98e1fc63874da7/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:21:12.875238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0a0f0-2b3a-43bd-9c46-8f24acb15d76 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.875845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0a0f0-2b3a-43bd-9c46-8f24acb15d76 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.875845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0a0f0-2b3a-43bd-9c46-8f24acb15d76 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.877155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 586/1154] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:12.882933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.883435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.883654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.901865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:12.903146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:12.985787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 569/1155] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:13.010217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0a0f0-2b3a-43bd-9c46-8f24acb15d76 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.106819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.120523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0a0f0-2b3a-43bd-9c46-8f24acb15d76 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.156466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.157253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.157535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.157810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.169385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:13.177089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:13.178946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:21:13.180583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 587/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:13.184840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.268668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.289320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deee573-dbaa-46ca-aba1-3a5ea86febf5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.324229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.325297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.325794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.326286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.327704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:13.328272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.330399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.340038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 570/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:13.346132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.451364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fd55f-1771-438b-bce0-a3ede781e07b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.486559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.487465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.487737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.488012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] PATH_INFO: `/v3/users/eafb76ad294443a5ab98e1fc63874da7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.489215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] RBAC: Authorizing `identity:get_user(user_id=eafb76ad294443a5ab98e1fc63874da7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.490660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.492329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 588/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/users/eafb76ad294443a5ab98e1fc63874da7 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:13.498430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.607484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b43dc5-a7d3-4f1c-b14f-e5af8f42e98f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.646055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.646725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.647005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.647306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.661788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.724227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.725702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.836433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 571/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:13.843500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.863107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.947766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b86f91-e5a2-4555-8e49-4c6282305835 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.981130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9dc79-7b77-4403-89c5-78628978dcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.981979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9dc79-7b77-4403-89c5-78628978dcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.982261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9dc79-7b77-4403-89c5-78628978dcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.982534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9dc79-7b77-4403-89c5-78628978dcc4 None admin] PATH_INFO: `/v3/projects/e6ca0d2625fb488e8634d7535a031ad4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.983697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9dc79-7b77-4403-89c5-78628978dcc4 None admin] RBAC: Authorizing `identity:get_project(project_id=e6ca0d2625fb488e8634d7535a031ad4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.985141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9dc79-7b77-4403-89c5-78628978dcc4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.986812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 589/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/projects/e6ca0d2625fb488e8634d7535a031ad4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:13.993141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:14.036488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:14.093860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d837c-5013-4a2c-bfe1-9c6ddcfe458b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:14.128973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:14.129532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:14.129810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:14.130082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:14.130913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e6ca0d2625fb488e8634d7535a031ad4, user.id=eafb76ad294443a5ab98e1fc63874da7) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:14.131247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:14.132680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7bd4c-72e3-492c-918f-707127b63df2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:14.139002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 572/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/role_assignments?user.id=eafb76ad294443a5ab98e1fc63874da7&scope.project.id=e6ca0d2625fb488e8634d7535a031ad4&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:16.282628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd1ca9-a43e-4a44-8d68-ecaeb51c88f3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.283220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd1ca9-a43e-4a44-8d68-ecaeb51c88f3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.283220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd1ca9-a43e-4a44-8d68-ecaeb51c88f3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.284591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 590/1162] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:16.290761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.291283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.291394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.308440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:16.309643 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:16.377626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 573/1163] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:16.401579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd1ca9-a43e-4a44-8d68-ecaeb51c88f3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.489990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.507353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd1ca9-a43e-4a44-8d68-ecaeb51c88f3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.543014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.543621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.543903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.544186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.620343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.621811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.709840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.726639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 591/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:21:16.732898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.767982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.833588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a177d-6cda-4b93-9f19-358274c32443 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.868382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.869179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.869452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.869721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.876906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.878350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6f3298fe-5b2b-46a4-95ed-fc275e2fd97e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:16.894013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:16.895495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:21:16.896749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 574/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:16.902027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:17.003703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce305-257e-4d61-b16e-4be92bf615e7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:17.039236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:17.040448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.040730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.041005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.041726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:17.042035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:17.043451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d0077-5236-4bbe-ac1a-74d7bf39caa6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:17.050984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 592/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:19.138011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0beb6-181c-4f6a-aa13-349017259172 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:19.138574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0beb6-181c-4f6a-aa13-349017259172 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:19.138574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0beb6-181c-4f6a-aa13-349017259172 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:19.140128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 575/1167] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:19.145514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb89d-0881-4c5b-abd3-644d6302c166 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:19.145917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb89d-0881-4c5b-abd3-644d6302c166 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:19.146083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb89d-0881-4c5b-abd3-644d6302c166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:19.162758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6d8cb89d-0881-4c5b-abd3-644d6302c166 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:19.163992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6d8cb89d-0881-4c5b-abd3-644d6302c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:19.206929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:19.230445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 593/1168] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:19.253553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0beb6-181c-4f6a-aa13-349017259172 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:19.348417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:19.355931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0beb6-181c-4f6a-aa13-349017259172 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:19.389378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45305878-ca11-41f3-b26a-e742f1a7c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:19.389967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305878-ca11-41f3-b26a-e742f1a7c590 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:19.390251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305878-ca11-41f3-b26a-e742f1a7c590 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:19.390529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305878-ca11-41f3-b26a-e742f1a7c590 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:19.391304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45305878-ca11-41f3-b26a-e742f1a7c590 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:19.392673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45305878-ca11-41f3-b26a-e742f1a7c590 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:19.407424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 576/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Thu Jul 21 22:21:19 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:21.468359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f7466-e5d1-4f4f-9b0b-33a6003b649d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:21.468948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f7466-e5d1-4f4f-9b0b-33a6003b649d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:21.468948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f7466-e5d1-4f4f-9b0b-33a6003b649d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:21.470267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 594/1170] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:21.475538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:21.475981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:21.476158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:21.493585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:21.494766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:21.563433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 577/1171] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:21.587779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f7466-e5d1-4f4f-9b0b-33a6003b649d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:21.690325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f7466-e5d1-4f4f-9b0b-33a6003b649d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:21.725136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce190f2-ee54-4b9a-a74b-7bfbc2a4ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:21.725922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce190f2-ee54-4b9a-a74b-7bfbc2a4ee3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:21.726208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce190f2-ee54-4b9a-a74b-7bfbc2a4ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:21.726490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce190f2-ee54-4b9a-a74b-7bfbc2a4ee3c None admin] PATH_INFO: `/v3/domains/55cc109427034df9a6dfa8a8dce27c1d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:21.736988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce190f2-ee54-4b9a-a74b-7bfbc2a4ee3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=55cc109427034df9a6dfa8a8dce27c1d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:21.738446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce190f2-ee54-4b9a-a74b-7bfbc2a4ee3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:21.740146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 595/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Jul 21 22:21:21 2022] GET /identity/v3/domains/55cc109427034df9a6dfa8a8dce27c1d => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:21.746394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:21.832105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:21.848121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773896c8-4c2e-492f-9a3f-097c351dd452 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:21.883786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe7377-c3ab-44d0-b800-67321209b14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:21.884453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe7377-c3ab-44d0-b800-67321209b14a None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:21.884738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe7377-c3ab-44d0-b800-67321209b14a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:21.885012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe7377-c3ab-44d0-b800-67321209b14a None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:21.885723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe7377-c3ab-44d0-b800-67321209b14a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:21.887185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe7377-c3ab-44d0-b800-67321209b14a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:21.912192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 578/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:21:21 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:23.987966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294332e-4a86-4a7e-9d9e-2f410414b7e4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.988487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294332e-4a86-4a7e-9d9e-2f410414b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.988487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294332e-4a86-4a7e-9d9e-2f410414b7e4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:23.989824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 596/1174] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:23.995299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.995751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.995850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:24.013662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:24.014840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:24.082597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 579/1175] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:24.108877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294332e-4a86-4a7e-9d9e-2f410414b7e4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:24.153464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:24.217402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294332e-4a86-4a7e-9d9e-2f410414b7e4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:24.254413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71079b-7043-4ba2-9285-9f0f663fd70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:24.255236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71079b-7043-4ba2-9285-9f0f663fd70f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:24.255522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71079b-7043-4ba2-9285-9f0f663fd70f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:24.255803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71079b-7043-4ba2-9285-9f0f663fd70f None admin] PATH_INFO: `/v3/domains/55cc109427034df9a6dfa8a8dce27c1d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:24.257018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71079b-7043-4ba2-9285-9f0f663fd70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=55cc109427034df9a6dfa8a8dce27c1d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:24.259261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71079b-7043-4ba2-9285-9f0f663fd70f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:24.260907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 597/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Jul 21 22:21:24 2022] GET /identity/v3/domains/55cc109427034df9a6dfa8a8dce27c1d => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:24.266758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:24.334062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:24.369284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df6425-23c1-484a-b474-3bcad5f261dd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:24.404002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd8255-f0ce-4d56-bc2a-07e192cf3a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:24.404673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8255-f0ce-4d56-bc2a-07e192cf3a86 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:24.404955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8255-f0ce-4d56-bc2a-07e192cf3a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:24.405228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8255-f0ce-4d56-bc2a-07e192cf3a86 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:24.405928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd8255-f0ce-4d56-bc2a-07e192cf3a86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:24.407412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd8255-f0ce-4d56-bc2a-07e192cf3a86 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:24.427671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 580/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:21:24 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:26.570791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03a44c-de01-4fca-8a87-676fb592fc98 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.571379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03a44c-de01-4fca-8a87-676fb592fc98 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.571379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03a44c-de01-4fca-8a87-676fb592fc98 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.572732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 598/1178] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:26.578611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.579121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.579285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.597159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:26.598536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:26.693472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 581/1179] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:26.720926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03a44c-de01-4fca-8a87-676fb592fc98 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:26.830750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03a44c-de01-4fca-8a87-676fb592fc98 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:26.839433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:26.870799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:26.871620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.871901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.872173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.887149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:26.895148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:26.896288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:21:26.897142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 599/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:26.904744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:26.968945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:27.019555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f4955-9813-4335-a550-a3b6ccf36029 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.060664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.061344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.061628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.062097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.062888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:27.063238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.064636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.074406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 582/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:27.080487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:27.191116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c189e-aab2-42ab-8d08-a0460003083d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.228903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.229780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.230057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.230340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] PATH_INFO: `/v3/users/c6eaa4614f384639803c596a2a7e6411` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.249034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] RBAC: Authorizing `identity:get_user(user_id=c6eaa4614f384639803c596a2a7e6411)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.250504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.252266 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 600/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:27 2022] GET /identity/v3/users/c6eaa4614f384639803c596a2a7e6411 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:27.258232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:27.368236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb373e-e4b2-44eb-8b51-d12c7fb127f4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.388444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:27.408873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.409547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.409836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.410120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.484854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.486242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.584981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 583/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:27.591702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:27.616482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:27.700051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed677d71-2c8e-478a-8fdf-a4a11a898a12 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.743081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c70e1-481b-48b6-9773-b24d656c6440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.743862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c70e1-481b-48b6-9773-b24d656c6440 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.744145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c70e1-481b-48b6-9773-b24d656c6440 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.744422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c70e1-481b-48b6-9773-b24d656c6440 None admin] PATH_INFO: `/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.745597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c70e1-481b-48b6-9773-b24d656c6440 None admin] RBAC: Authorizing `identity:get_project(project_id=ac3b8f3fb2b04017a40c315089de0bc4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.747045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c70e1-481b-48b6-9773-b24d656c6440 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.748768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 601/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:27 2022] GET /identity/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:27.754978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:27.774038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:27.856468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc5c3-4614-4556-8d2a-e9257737a4b7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.890691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.891373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.891660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.891938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.892782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=ac3b8f3fb2b04017a40c315089de0bc4, user.id=c6eaa4614f384639803c596a2a7e6411) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:27.893098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.894790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc241de-fbca-4258-a311-f511baa2f2a7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.901089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 584/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:27 2022] GET /identity/v3/role_assignments?user.id=c6eaa4614f384639803c596a2a7e6411&scope.project.id=ac3b8f3fb2b04017a40c315089de0bc4&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:29.913521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a43bb-b2c1-4f51-8851-30841ffd1046 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.914032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a43bb-b2c1-4f51-8851-30841ffd1046 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.914032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a43bb-b2c1-4f51-8851-30841ffd1046 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.915395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 602/1186] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:29.920791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.921236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.921351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.938035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:29.939251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:30.004834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 585/1187] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:30.029821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a43bb-b2c1-4f51-8851-30841ffd1046 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.121047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.130857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a43bb-b2c1-4f51-8851-30841ffd1046 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.166801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.167545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.167833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.168117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.179810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:30.188111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:30.189384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:21:30.190409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 603/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:30.196433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.208200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.297909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ab453-ba61-46f3-9723-6516dccd8093 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.331973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.332642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.332921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.333196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.333949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:30.334253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:30.335805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.344091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 586/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:30.350200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.459474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cb2dc-3b34-448a-b102-d01a60227f39 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.495463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.496344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.496621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.496892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] PATH_INFO: `/v3/users/c6eaa4614f384639803c596a2a7e6411` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.498125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] RBAC: Authorizing `identity:get_user(user_id=c6eaa4614f384639803c596a2a7e6411)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:30.499616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.501277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 604/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:30 2022] GET /identity/v3/users/c6eaa4614f384639803c596a2a7e6411 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:30.507363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.583163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.618363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a725c4-5376-42ce-8d58-45f0b43c59a2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.652954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a2c6e-e5d3-427e-8dd7-c2eded2e8916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.653817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a2c6e-e5d3-427e-8dd7-c2eded2e8916 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.654105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a2c6e-e5d3-427e-8dd7-c2eded2e8916 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.654380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a2c6e-e5d3-427e-8dd7-c2eded2e8916 None admin] PATH_INFO: `/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.655608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a2c6e-e5d3-427e-8dd7-c2eded2e8916 None admin] RBAC: Authorizing `identity:get_project(project_id=ac3b8f3fb2b04017a40c315089de0bc4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:30.657049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a2c6e-e5d3-427e-8dd7-c2eded2e8916 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.658787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 587/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:30 2022] GET /identity/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:30.665342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.687721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.767945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1203a9-ebd3-467f-9f89-719890667d2a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.803940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372f477-5b82-45b4-8678-ca345cb1494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.805797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372f477-5b82-45b4-8678-ca345cb1494f None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.806082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372f477-5b82-45b4-8678-ca345cb1494f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.806358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372f477-5b82-45b4-8678-ca345cb1494f None admin] PATH_INFO: `/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4/users/c6eaa4614f384639803c596a2a7e6411/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.808334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372f477-5b82-45b4-8678-ca345cb1494f None admin] RBAC: Authorizing `identity:create_grant(project_id=ac3b8f3fb2b04017a40c315089de0bc4, user_id=c6eaa4614f384639803c596a2a7e6411, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:30.809789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372f477-5b82-45b4-8678-ca345cb1494f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.822945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 605/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:21:30 2022] PUT /identity/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4/users/c6eaa4614f384639803c596a2a7e6411/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:21:32.864671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f63c6-c77c-4c62-8a86-4444745100c4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:32.865233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f63c6-c77c-4c62-8a86-4444745100c4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:32.865233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f63c6-c77c-4c62-8a86-4444745100c4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:32.867219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 588/1193] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:32.872585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:32.873016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:32.873136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:32.890233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:32.891484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:32.964954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 606/1194] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:32.988976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f63c6-c77c-4c62-8a86-4444745100c4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:33.096991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f63c6-c77c-4c62-8a86-4444745100c4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:33.131013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:33.131830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:33.132110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:33.132402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:33.140045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:33.144053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:33.152182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:33.153309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:21:33.154371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 589/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:33.159511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:33.172432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:33.263675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a31df1-b561-4bce-aeae-1cb776b030b4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:33.299377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:33.300039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:33.300322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:33.300599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:33.301342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:33.301648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:33.303029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:33.311430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 607/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:33.317403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:33.416811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee412bb-3b92-43e0-afe5-997efdf3e569 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:33.451320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:33.452179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:33.452456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:33.452726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] PATH_INFO: `/v3/users/c6eaa4614f384639803c596a2a7e6411` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:33.453893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] RBAC: Authorizing `identity:get_user(user_id=c6eaa4614f384639803c596a2a7e6411)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:33.455347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:33.456973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 590/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:33 2022] GET /identity/v3/users/c6eaa4614f384639803c596a2a7e6411 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:33.463165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:33.531968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:33.561850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e89a6-01b8-49e0-bfcc-a9d03f90bf1d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:33.595101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:33.595751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:33.596037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:33.596323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:33.666379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:33.667892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:33.735269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:33.766055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 608/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:33.772355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:33.837825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:33.871598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7256af-64f0-45a1-9e52-19eabf270612 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:33.905948 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754819d-8091-4509-a932-af93908f6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:33.906802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754819d-8091-4509-a932-af93908f6997 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:33.907110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754819d-8091-4509-a932-af93908f6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:33.907390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754819d-8091-4509-a932-af93908f6997 None admin] PATH_INFO: `/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:33.908557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754819d-8091-4509-a932-af93908f6997 None admin] RBAC: Authorizing `identity:get_project(project_id=ac3b8f3fb2b04017a40c315089de0bc4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:33.909999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754819d-8091-4509-a932-af93908f6997 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:33.911738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 591/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:33 2022] GET /identity/v3/projects/ac3b8f3fb2b04017a40c315089de0bc4 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:33.918502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:34.015137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae355191-051b-45f3-b65a-17a41f75df3b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:34.051685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:34.052336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:34.052617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:34.052892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:34.053708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=ac3b8f3fb2b04017a40c315089de0bc4, user.id=c6eaa4614f384639803c596a2a7e6411) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:34.054044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:34.055584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fde0e-5910-4342-bd4a-f61469d2ab28 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:34.061381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 609/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:33 2022] GET /identity/v3/role_assignments?user.id=c6eaa4614f384639803c596a2a7e6411&scope.project.id=ac3b8f3fb2b04017a40c315089de0bc4&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:43.901691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=32 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:44.058807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=82 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:31.146219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba70576-c862-4f14-9446-0b8f2b4d1061 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:31.146823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba70576-c862-4f14-9446-0b8f2b4d1061 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:31.146823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba70576-c862-4f14-9446-0b8f2b4d1061 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:31.148303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 592/1201] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:31.154103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:31.154678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:31.154678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:31.173623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:31.174844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:31.267109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 610/1202] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:22:31.292403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba70576-c862-4f14-9446-0b8f2b4d1061 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:31.435581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba70576-c862-4f14-9446-0b8f2b4d1061 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:31.489356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:31.490241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:31.490543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:31.493907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:31.504691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:31.506300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:31.526663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:31.527952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:31.528937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:31.528937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:31.528937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:31.528937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:31.530688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 593/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:31.537257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:31.686504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1794-7a6f-4e79-8e2b-858104ac39bb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:31.728232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:31.736761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:31.737466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:31.738299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:31.738570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:31.739347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:31.739660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:31.741064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588c81b-a810-4ce9-bf70-b9f9ea652a0a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:31.761658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 611/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:31.762926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:31.828045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:31.890503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3fee1-c473-45c2-8494-ca4a60daaa9e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:31.945476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b88609-cdd5-422e-a3ea-ba2ddd8f5990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:31.946127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b88609-cdd5-422e-a3ea-ba2ddd8f5990 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:31.946465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b88609-cdd5-422e-a3ea-ba2ddd8f5990 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:31.946772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b88609-cdd5-422e-a3ea-ba2ddd8f5990 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:31.947553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b88609-cdd5-422e-a3ea-ba2ddd8f5990 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:31.949138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b88609-cdd5-422e-a3ea-ba2ddd8f5990 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:31.980501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 594/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jul 21 22:22:31 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:22:34.303983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f424c-78e4-4878-9155-5b29bc1bd68c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:34.304549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f424c-78e4-4878-9155-5b29bc1bd68c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:34.304549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f424c-78e4-4878-9155-5b29bc1bd68c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:34.306055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 612/1206] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:34.312404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:34.312899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:34.312987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:34.331675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:34.333098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:34.421737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 595/1207] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:34.450742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f424c-78e4-4878-9155-5b29bc1bd68c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:34.572004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f424c-78e4-4878-9155-5b29bc1bd68c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:34.610759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:34.611609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:34.611895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:34.612210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:34.625773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:34.635476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:34.636554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:34.638468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 613/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:22:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:34.643606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:34.691941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:34.776361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1df3a7-f603-448d-9cee-f6c4c4cde3b6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:34.825185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:34.825901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:34.826260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:34.826561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:34.827432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:34.827744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:34.829294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:34.838943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 596/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:22:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:34.844943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:34.907706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:34.981310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0166-8753-4b50-8384-193df450401c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:35.025418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:35.026513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:35.026894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:35.027332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:35.043388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:35.045087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:35.065069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:35.067142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:35.072889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:35.072889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:35.072889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:35.072889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:35.072889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 614/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:35.074144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:35.203028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:35.208130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744dc8-5fc3-4c03-9d80-da8d9a40dc87 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:35.252119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:35.252979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:35.253879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:35.254294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:35.255173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:35.255487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:35.257200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:35.272929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 597/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:35.279530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:35.399753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000be198-b168-49d4-8da3-5fd16d1423af None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:35.403465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:35.440937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:35.442292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:35.442583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:35.442863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:35.451460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:35.455371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:22:35.473078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:35.474182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:35.475188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:35.479659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 615/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:35.480165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:35.601133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15f862-3444-4751-9f28-9803744374bb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:35.637162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:35.637838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:35.638115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:35.638411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:35.639266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:35.639576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:35.643199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:35.661629 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 598/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jul 21 22:22:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:35.666366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:35.790552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7bb14-0fc6-4c8f-af09-234980cf525d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:35.835111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:35.835111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:35.835111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:35.835111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:35.869476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:35.919469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:35.920976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:36.059111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 616/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:22:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:22:36.064834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:36.086178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:36.187409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52275f3b-fd85-4915-abb6-0e81cf779d71 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:36.226451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:36.227507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:36.227908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:36.228223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:36.241147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:36.242664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:36.265695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:36.267312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:36.268538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:36.268538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:36.268538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:36.268538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:36.274161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 599/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:36.274370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:36.349140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:36.395387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278031d-43e2-41b4-95c9-167b06008c5c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:36.434305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:36.435100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:36.435457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:36.435763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:36.436543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:36.436901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:36.438538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:36.452434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 617/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:22:36.458297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:36.544116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:36.578949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e759-f062-47ab-93e5-804ad2238273 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:36.619527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:36.620301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:36.620612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:36.620901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:36.629132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:36.630602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:22:36.647782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:36.648834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:36.649750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 600/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:36.655294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:36.793958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62956b19-0b39-4043-a976-dafe29231fd1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:36.821705 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:36.836133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:36.836830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:36.837127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:36.837400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:36.838222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:36.838544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:36.840178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e25c43-5137-4edc-b87b-403372bf28ae None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:36.860964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 618/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:22:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:36.862264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:36.990876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d445fe-7fbc-42dd-a4d5-574cefbdb1c7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:37.036371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:37.047418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:37.048298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:37.048698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:37.048973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:37.049862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=019c423e880647139535f2d0dfc5a9c5) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:37.050458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:37.052403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0db06e-f1d0-4408-867f-9d15b019a60c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:37.071091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 601/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:22:36 2022] GET /identity/v3/role_assignments?user.id=019c423e880647139535f2d0dfc5a9c5&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:39.257022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daa713-e77f-49a4-9a89-481ef946850c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:39.259028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daa713-e77f-49a4-9a89-481ef946850c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:39.259808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daa713-e77f-49a4-9a89-481ef946850c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:39.263103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 619/1220] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:39.269453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:39.269453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:39.270151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:39.291034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:39.292277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:39.369735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 602/1221] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:39.396375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daa713-e77f-49a4-9a89-481ef946850c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:39.516990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daa713-e77f-49a4-9a89-481ef946850c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:39.558264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:39.559087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:39.559697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:39.560060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:39.572236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:39.581461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:39.582586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:39.583764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 620/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:22:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:39.589227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:39.711120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6bc0-e56f-44d4-82a1-0107ee5bf987 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:39.741533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:39.755692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:39.756468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:39.756687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:39.757002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:39.757758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:39.758065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:39.759463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:39.770309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 603/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:22:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:39.776279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:39.825194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:39.890177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1307e85-e3aa-403b-bbac-00d9f28c0426 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:39.931337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:39.932227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:39.932497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:39.932853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:39.944993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:39.946686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:39.966156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:39.967370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:39.968510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:39.968510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:39.968510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:39.968510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:39.971201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 621/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:39.974022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:40.087324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9215d-7a51-4ed3-85d2-81143aff7aef None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:40.129625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:40.130295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:40.130583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:40.130860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:40.131625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:40.131937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:40.133317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:40.145758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 604/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:40.151869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:40.247444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:40.257329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc99c-bf41-4573-ab7d-8c35c77975e1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:40.298096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:40.298829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:40.299151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:40.299434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:40.306957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:40.308459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:22:40.324866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:40.325920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:40.326893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 622/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:40.332848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:40.387537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:40.451289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d21de-9fa8-46bd-ae75-1afb9eb1ab8e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:40.485877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:40.486542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:40.486825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:40.487191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:40.488189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:40.488498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:40.490334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:40.512188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 605/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jul 21 22:22:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:40.519144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:40.628888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d4bd5-56a7-4126-8090-c7d5ee9421c0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:40.666801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:40.667670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:40.667992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:40.668265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:40.679742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:40.681204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:40.699398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:40.700392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:40.701338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:40.701338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:40.701338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:40.701338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:40.701338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 623/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:40.706900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:40.820835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d678b2-1891-4337-becc-b88def736656 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:40.823169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:40.863542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:40.864287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:40.864568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:40.864839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:40.865561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:40.865860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:40.867459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:40.880806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 606/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:40.887223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:40.899451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:41.016678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfb02c-cabe-4c04-aeaa-6af64a153ce5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:41.058366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:41.059307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:41.059588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:41.059859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:41.067806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:41.069364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:22:41.085724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:41.086874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:41.088036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 624/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:22:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:41.093148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:41.209474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cd3f0-23c6-49c6-a08a-b5c5f3bf12c6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:41.248463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:41.249219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:41.249522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:41.249854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:41.250783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:41.251130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:41.253251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a662ed2-b6af-4a0c-9a54-2afdc7adacf3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:41.271036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 607/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:22:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:22:41.277798 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:41.357441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:41.406193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac77b9-2f98-4119-af5a-5d68594e7934 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:41.455089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d47b0-386c-46be-8f8a-e0288136b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:41.456986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d47b0-386c-46be-8f8a-e0288136b6ff None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:41.457246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d47b0-386c-46be-8f8a-e0288136b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:41.457606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d47b0-386c-46be-8f8a-e0288136b6ff None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/019c423e880647139535f2d0dfc5a9c5/roles/683781dab2524c4fb9a282ca9270dc0c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:41.475983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d47b0-386c-46be-8f8a-e0288136b6ff None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=019c423e880647139535f2d0dfc5a9c5, role_id=683781dab2524c4fb9a282ca9270dc0c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:41.477643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d47b0-386c-46be-8f8a-e0288136b6ff None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:41.497232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 625/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:22:41 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/019c423e880647139535f2d0dfc5a9c5/roles/683781dab2524c4fb9a282ca9270dc0c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:22:43.711798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510718c9-5332-4836-aefa-f1dbf89d4297 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:43.712340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510718c9-5332-4836-aefa-f1dbf89d4297 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:43.712728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510718c9-5332-4836-aefa-f1dbf89d4297 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:43.715144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 608/1233] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:43.721117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:43.722278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:43.722685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:43.743197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:43.744428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:43.821168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 626/1234] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:43.847631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510718c9-5332-4836-aefa-f1dbf89d4297 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:43.876332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:43.966991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510718c9-5332-4836-aefa-f1dbf89d4297 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:44.006620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:44.007465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:44.007745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:44.008016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:44.028131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:44.037202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:44.038413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:44.044319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 609/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jul 21 22:22:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:44.045213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:44.105240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:44.171084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43457fe-ee69-4e9e-adaa-f6f07e352778 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:44.210526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:44.211392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:44.211686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:44.212006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:44.212964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:44.213279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:44.214840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:44.225781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 627/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jul 21 22:22:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:44.232036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:44.342020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:44.366499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6975504-bb82-42fe-b1b5-93c965b88091 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:44.415572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:44.416392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:44.416677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:44.416955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:44.432007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:44.433496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:44.452144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:44.453603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:44.454865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:44.454865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:44.454865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:44.454865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:44.454865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 610/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:44.460399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:44.578401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dfda4-cf4c-4dd5-ac9b-cdfead5ad428 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:44.591210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:44.621098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:44.621784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:44.622061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:44.622337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:44.623098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:44.623400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:44.624821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:44.638252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 628/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:44.646153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:44.767284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e19cb-d5d6-4d7f-9ac7-3b75562678b3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:44.806280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:44.807107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:44.807395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:44.807671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:44.810911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:44.818954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:44.820466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:22:44.837693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:44.838890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:44.840082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 611/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:44.845610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:44.957164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed362a-a611-40fd-8f14-570e31d49162 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:44.996352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:44.997031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:44.997318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:44.997598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:44.998465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:44.998809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:45.000399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:45.021140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 629/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jul 21 22:22:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:45.028103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:45.151315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58cd02-7b1a-409e-94c9-1f2ff431c64a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:45.197550 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:45.198744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:45.199315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:45.199789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:45.283144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:45.296450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:45.297823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:45.406813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 612/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:22:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:22:45.413511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:45.444959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:45.526475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bc36-fe5e-4e11-818d-e25df91a46a0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:45.568170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:45.569043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:45.569336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:45.569621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:45.581867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:45.583498 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:45.602880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:45.603922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:45.604820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:45.604820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:45.604820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:45.604820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:45.604820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 630/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:45.610244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:45.717213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:45.725108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9440a8-e147-4f6f-aef3-f9253e64b83d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:45.763996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:45.764776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:45.765099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:45.765410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:45.766400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:45.766712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:45.768210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:45.781297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 613/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:45.787932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:45.881783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:45.904916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d109c-fb8f-41ed-a65d-d7f4ee9f6e1f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:45.944061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:45.944974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:45.945285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:45.945638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:45.953314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:45.954778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:22:45.969820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:45.970977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:45.973108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 631/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:22:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:45.977469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:46.093280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26326c76-12af-476f-b21a-0fcbe44c2713 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:46.132833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:46.133509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:46.133791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:46.134160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:46.135136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:46.135540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:46.137359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647482b0-0dc5-4cf8-ad1c-111bba277093 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:46.157645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 614/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:22:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:46.159208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:46.288018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b1537-596f-4247-9ce5-12537c6288af None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:46.333925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:46.334587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:46.334876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:46.335181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:46.336237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] RBAC: Adding query filter params (role.id=683781dab2524c4fb9a282ca9270dc0c, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=019c423e880647139535f2d0dfc5a9c5) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:46.336677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:46.338382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a28d07-b8cf-4068-b616-5a57c8180c18 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:46.345520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 632/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:22:46 2022] GET /identity/v3/role_assignments?user.id=019c423e880647139535f2d0dfc5a9c5&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=683781dab2524c4fb9a282ca9270dc0c => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:48.709487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74296eda-110c-4dd8-9265-4e4770ba640f None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:48.710054 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74296eda-110c-4dd8-9265-4e4770ba640f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:48.710054 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74296eda-110c-4dd8-9265-4e4770ba640f None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:48.711503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 615/1247] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:48.716969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:48.717428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:48.717538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:48.722889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:48.735442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:48.736666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:48.810951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 633/1248] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:48.838904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74296eda-110c-4dd8-9265-4e4770ba640f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:48.861304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:48.959330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74296eda-110c-4dd8-9265-4e4770ba640f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:48.998155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:48.998956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:48.999280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:48.999554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:49.012808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:49.021389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:49.022705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:49.023763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 616/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:22:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:49.028919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:49.136079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d24501-3ff7-4d3c-a88a-40a9e77c062b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:49.172468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:49.178552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:49.179270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:49.179555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:49.179828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:49.180578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:49.180886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:49.182379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:49.192864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 634/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:22:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:49.199632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:49.299656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:49.329122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785a89f-3109-468b-ba38-0d3053561a52 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:49.370717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:49.371674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:49.371963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:49.372243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:49.385130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:49.386600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:49.406082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:49.407170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:49.408079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:49.408079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:49.408079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:49.408079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:49.408079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 617/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:49.414248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:49.538606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c67c1-08b2-4be4-af03-c14e3ef18ba0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:49.584131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:49.584794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:49.585082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:49.585411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:49.586201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:49.586487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:49.588090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:49.604125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 635/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:49.610443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:49.738479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f6b5-66dd-42b3-a652-fdc16f365639 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:49.769173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:49.777665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:49.778549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:49.778835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:49.779145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:49.787655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:49.789215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:22:49.805154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:49.806388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:49.807405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 618/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:49.812385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:49.876271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:49.932859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410429-7b3d-4bbd-93d5-dc6bb9d7867d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:49.975583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:49.976343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:49.976675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:49.976993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:49.977922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:49.978239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:49.979819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:50.001263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 636/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jul 21 22:22:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:50.008894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:50.134223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f23e2-a5fc-4d43-8ed7-c565abe15e64 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:50.180690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:50.181437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:50.181776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:50.182011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:50.251317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:50.270419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:50.272147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:50.402170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 619/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:22:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:22:50.409176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:50.543594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15806d6-f2a9-448d-88a5-1dc1952d58ce None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:50.566564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:50.604955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:50.605823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:50.606175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:50.606511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:50.621190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:50.622738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:50.643982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:50.645159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:50.647101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:50.647101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:50.647101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:50.647101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:50.647101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 637/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:50.651723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:50.756499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:50.776048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7f43-1b9f-4fe5-b70f-fcc97efbcfc6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:50.819775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:50.820458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:50.820742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:50.821016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:50.821751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:50.822059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:50.823481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:50.838211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 620/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:50.844637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:50.974978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bc434-c103-4b7c-a5d3-0a44a64346eb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:51.017305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:51.018940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:51.019276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:51.019562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:51.029997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:51.032116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:22:51.050211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:51.051385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:51.052302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:51.053106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 638/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:51.058774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:51.185893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3bf6d-54cd-4831-9e2d-2fb82a4811da None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:51.225960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:51.226541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:51.226834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:51.227145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:51.227945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:51.228258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:51.229705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba23072-6359-4984-a866-9f74da0f11aa None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:51.243535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 621/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:22:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:51.250231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:51.284414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:51.390076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42b8e7-9dae-4afb-b9d6-a400e0222c36 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:51.430694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:51.431405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:51.431694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:51.431971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:51.432814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=019c423e880647139535f2d0dfc5a9c5) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:51.433133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:51.434695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f908df-4f3b-42e3-ac2d-288e28d96ae6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:51.441177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 639/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:22:51 2022] GET /identity/v3/role_assignments?user.id=019c423e880647139535f2d0dfc5a9c5&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:53.520789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aba7c6-c5cd-431a-ba7b-d8e91264d7a8 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:53.521357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aba7c6-c5cd-431a-ba7b-d8e91264d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:53.521619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aba7c6-c5cd-431a-ba7b-d8e91264d7a8 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:53.523275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 622/1261] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:53.528959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:53.529414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:53.529542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:53.551003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:53.552791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:53.619126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 640/1262] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:53.641889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aba7c6-c5cd-431a-ba7b-d8e91264d7a8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:53.653374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:53.743171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aba7c6-c5cd-431a-ba7b-d8e91264d7a8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:53.776250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:53.776969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:53.777252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:53.777530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:53.788405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:53.796073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:53.797189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:53.798257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 623/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:53.803593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:53.822006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:53.902051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40339efe-0f5f-4f60-8427-f71011f43198 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:53.936352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:53.937013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:53.937295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:53.937566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:53.938308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:53.938614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:53.940186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:53.948501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 641/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:53.954394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:54.019330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:54.052523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f88df-fd8e-4516-95f1-cd42905432eb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:54.084409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:54.085192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:54.085478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:54.085747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:54.096641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:54.098079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:54.115259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:54.116369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:54.117439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:54.117439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:54.117439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:54.117439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:54.117439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 624/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:54.122563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:54.202047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:54.220261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4637f4-37d1-412f-84ab-f49915201d51 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:54.254324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:54.254963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:54.255271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:54.255597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:54.256331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:54.256760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:54.258083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:54.272029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 642/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:54.278695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:54.387816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a4936-ff07-4ab8-8b1e-9afe10f606cf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:54.407494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:54.423859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:54.424563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:54.424942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:54.425229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:54.435128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:54.436625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:22:54.452854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:54.454115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:54.455749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 625/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:54.461470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:54.570888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e777-5fcf-44a0-bd62-76fd3161db03 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:54.607068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:54.607769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:54.608054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:54.608330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:54.609184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:54.609500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:54.610963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:54.620636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:54.630577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 643/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jul 21 22:22:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:54.636398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:54.742161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c2116-73f4-4110-a34d-99ce870f9d18 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:54.780804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:54.781947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:54.782378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:54.782788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:54.794266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:54.795761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:54.813268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:54.814849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:54.816029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:54.816029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:54.816029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:54.816029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:54.816029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 626/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:54.820855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:54.920598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a31888-0481-4154-9fdb-485edf24e33e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:54.957841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:54.958523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:54.958808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:54.959105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:54.959833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:54.960139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:54.961520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:54.973239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 644/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:54.979717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:55.012894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:55.084503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982bf43-2ba7-4575-8f15-fb166b88a857 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:55.116879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:55.117664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:55.117947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:55.118224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:55.124698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:55.126149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:22:55.141246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:55.142316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:55.143609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 627/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:22:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:55.148331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:55.205786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:55.248686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b65f7-a6cc-45c2-a3f5-cec8817a5869 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:55.293787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:55.294473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:55.294757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:55.295034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:55.295848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:55.296158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:55.297599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2218d-07bb-41d6-ab32-ce90b6e0d502 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:55.309613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 645/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:22:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:55.316292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:55.405580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:55.414753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40938c4-3fe9-4dfa-bf2b-84437dd44511 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:55.448903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820ee81-81ff-4a1b-a8d5-0c3c2c8ab863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:55.449789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820ee81-81ff-4a1b-a8d5-0c3c2c8ab863 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:55.450075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820ee81-81ff-4a1b-a8d5-0c3c2c8ab863 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:55.450350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820ee81-81ff-4a1b-a8d5-0c3c2c8ab863 None admin] PATH_INFO: `/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/019c423e880647139535f2d0dfc5a9c5/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:55.452356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820ee81-81ff-4a1b-a8d5-0c3c2c8ab863 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9e65f853d5348279e8ec6b19ee9773a, user_id=019c423e880647139535f2d0dfc5a9c5, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:55.453791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820ee81-81ff-4a1b-a8d5-0c3c2c8ab863 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:55.467746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 628/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jul 21 22:22:55 2022] PUT /identity/v3/projects/e9e65f853d5348279e8ec6b19ee9773a/users/019c423e880647139535f2d0dfc5a9c5/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:22:57.492679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314eded-1eed-45e4-a79a-3bc769d8927d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:57.493241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314eded-1eed-45e4-a79a-3bc769d8927d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:57.493241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314eded-1eed-45e4-a79a-3bc769d8927d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:57.494603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 646/1274] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:57.499885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:57.500352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:57.500447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:57.517973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:57.519951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:57.606432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 629/1275] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:57.632819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314eded-1eed-45e4-a79a-3bc769d8927d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:57.749867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314eded-1eed-45e4-a79a-3bc769d8927d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:57.771855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:57.789484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:57.790261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:57.790549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:57.790827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:57.801481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:57.809304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:57.810396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:57.811546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 647/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jul 21 22:22:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:57.818014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:57.906983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:57.928401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c030d77-1172-4e2e-a72f-747b67efe261 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:57.965964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:57.966638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:57.966921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:57.967247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:57.968077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:57.968615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:57.970194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:57.979661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 630/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jul 21 22:22:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:57.985415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:58.097804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd35fa-1aaf-4f36-9228-9bef17a8bcd5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:58.133780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b795b-dfff-4148-8334-53549289bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:58.134593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b795b-dfff-4148-8334-53549289bdee None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:58.134881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b795b-dfff-4148-8334-53549289bdee None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:58.135191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b795b-dfff-4148-8334-53549289bdee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:58.147077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b795b-dfff-4148-8334-53549289bdee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:58.148570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b795b-dfff-4148-8334-53549289bdee None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-586b795b-dfff-4148-8334-53549289bdee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:58.166934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:58.168626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:58.170684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:58.170684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:58.170684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:58.170684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:58.170684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 648/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:58.178318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:58.315232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def518e-4d56-4008-8fa4-a6de289e93a5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:58.379241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:58.380279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:58.380753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:58.381244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:58.382586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:58.383249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:58.385732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:58.409449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 631/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:58.415585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b795b-dfff-4148-8334-53549289bdee None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:58.451351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:58.530180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b795b-dfff-4148-8334-53549289bdee None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:58.566894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:58.567763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:58.568049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:58.568328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:58.576481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:58.578011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:22:58.593973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:58.595843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:22:58.597595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 649/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:58.602733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:58.606801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:58.769793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07431702-5e91-41e7-83b3-467ac1a9ab64 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:58.827740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:58.828864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:58.829351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:58.829844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:58.831556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:58.832100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:58.834695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:58.866462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 632/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jul 21 22:22:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:22:58.872791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:59.027918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:59.043942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa38d59-6fd0-4726-befb-b094771c2d21 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:59.098157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:59.098815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:59.099125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:59.099405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:59.174217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:59.175581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:59.286396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:59.298482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 650/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jul 21 22:22:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:22:59.305806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:59.373189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:59.416396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8792ea-adbd-463d-aeb5-a42788096782 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:59.456435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:59.457240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:59.457521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:59.457799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:59.469292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:59.471476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:22:59.491242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:59.493027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:59.495095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:22:59.495095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:22:59.495095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:22:59.495095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:59.495095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 633/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:59.499357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:59.616010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef14ef7-07eb-4d22-b966-40b76a8ad6dc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:59.653127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:59.653805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:59.654093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:59.654369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:59.655143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:59.655460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:59.656856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:59.671443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 651/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jul 21 22:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:59.678164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:59.848265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b9c5f-a51c-4e0f-93fa-793336632ea0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:59.874730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:59.908653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:59.909984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:59.910464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:59.910912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:59.920562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:59.922688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:22:59.948776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:59.949814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:22:59.950701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 21 22:22:59.951184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 634/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jul 21 22:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:22:59.958065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:00.009648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:00.090084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db555442-ff24-4432-aa43-af833f2339fd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:00.133816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:00.134411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:00.134701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:00.134984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:00.135849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:00.136165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:00.137669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880af26-19bf-4783-8160-ee0ad013feea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:00.152815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 652/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jul 21 22:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:00.159624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:00.292416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648a334-3882-4299-949c-f9dee2f7135a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:00.327500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:00.328152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:00.328432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:00.328705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:00.329541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] RBAC: Adding query filter params (role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67, scope.project.id=e9e65f853d5348279e8ec6b19ee9773a, user.id=019c423e880647139535f2d0dfc5a9c5) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:00.329880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:00.331347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43804-b8a1-40aa-8ad7-73c2a6d2d302 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:00.337978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 635/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jul 21 22:23:00 2022] GET /identity/v3/role_assignments?user.id=019c423e880647139535f2d0dfc5a9c5&scope.project.id=e9e65f853d5348279e8ec6b19ee9773a&role.id=dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:02.552962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ca247-01d3-483a-8a8a-0322881e6cc4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:02.553511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ca247-01d3-483a-8a8a-0322881e6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:02.553602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ca247-01d3-483a-8a8a-0322881e6cc4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:02.555090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 653/1288] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:02.560700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:02.561183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:02.561282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:02.582075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:02.583365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:02.619709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:02.665417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 636/1289] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:02.691120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ca247-01d3-483a-8a8a-0322881e6cc4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:02.811644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ca247-01d3-483a-8a8a-0322881e6cc4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:02.827077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:02.859211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:02.860151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:02.860591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:02.860970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:02.861721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:02.863237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:02.875625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:02.878237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:02.880069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 654/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:23:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:02.884352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:03.015903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e0fd-c082-403f-a462-cfd7bde3d2df None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:03.055704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:03.056510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:03.056794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:03.057074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:03.057794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:03.058105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:03.060144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa144-b0aa-4270-ba4a-5bbdd56c8ec2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:03.067857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 637/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:03.075832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:03.200125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae4757-a4e8-4ee1-bf74-6c102d9e212f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:03.244172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:03.244850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:03.245137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:03.245417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:03.246420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:03.246728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:03.248915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cbdcb-791c-4d73-a47c-ee2cef979e81 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:03.258435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 655/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:05.485055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cdf2b-dc14-4237-9e92-f82379589e4b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:05.485729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cdf2b-dc14-4237-9e92-f82379589e4b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:05.485729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cdf2b-dc14-4237-9e92-f82379589e4b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:05.487148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 638/1293] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:05.493263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48663ed-812c-4d4c-8923-831f2f38f8f8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:05.493749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48663ed-812c-4d4c-8923-831f2f38f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:05.494110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48663ed-812c-4d4c-8923-831f2f38f8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:05.520735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c48663ed-812c-4d4c-8923-831f2f38f8f8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:05.522489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c48663ed-812c-4d4c-8923-831f2f38f8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:05.571263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:05.617294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 656/1294] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:05.644668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cdf2b-dc14-4237-9e92-f82379589e4b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:05.682169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:05.776555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cdf2b-dc14-4237-9e92-f82379589e4b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:05.819233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c897484-bfdd-4fc3-b32e-66c6d69c55b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:05.819921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897484-bfdd-4fc3-b32e-66c6d69c55b2 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:05.820202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897484-bfdd-4fc3-b32e-66c6d69c55b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:05.820474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897484-bfdd-4fc3-b32e-66c6d69c55b2 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:05.821312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c897484-bfdd-4fc3-b32e-66c6d69c55b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:05.822871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c897484-bfdd-4fc3-b32e-66c6d69c55b2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:05.845840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 639/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jul 21 22:23:05 2022] POST /identity/v3/services => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:23:08.076826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec59aae-17c4-45b7-8384-4839d1a1300b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.077394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec59aae-17c4-45b7-8384-4839d1a1300b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.077394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec59aae-17c4-45b7-8384-4839d1a1300b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.078805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 657/1296] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:08.085686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.086366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.086575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.106112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:08.107399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:08.194066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 640/1297] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:08.220356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec59aae-17c4-45b7-8384-4839d1a1300b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.374590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec59aae-17c4-45b7-8384-4839d1a1300b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.411342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.412145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.412425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.412698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.413359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.414739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:08.428486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:08.430426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:08.432145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 658/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:08.438048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.457623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:08.546849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8817-03e4-4ccf-ab5a-0ec94dde9b9f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.584127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.584802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.585089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.585369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.586091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:08.586399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.587822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.593501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 641/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:08.600730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.637731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:08.731274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9bf3-0a3e-4260-8282-e6fe15e46720 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.770047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.770708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.770991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.771294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.772186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] RBAC: Adding query filter params (interface=public, service_id=ee78d27894184c58b7ca622b303ffdcb, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:08.772495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.773885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0238c8-2b83-4566-a603-54d5a4a22bac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.779887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 659/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/endpoints?service_id=ee78d27894184c58b7ca622b303ffdcb&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:08.788240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.879421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:08.900197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfab3f-1304-467a-8ff6-c4be86a7d871 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.939735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cc40c-6ed3-4041-8573-0fb00b41d9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.940405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cc40c-6ed3-4041-8573-0fb00b41d9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.940691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cc40c-6ed3-4041-8573-0fb00b41d9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.940971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cc40c-6ed3-4041-8573-0fb00b41d9f8 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.941666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cc40c-6ed3-4041-8573-0fb00b41d9f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.943093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cc40c-6ed3-4041-8573-0fb00b41d9f8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.950479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 642/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/services => generated 2191 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:23:11.193057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7848d9-40a5-4eaf-81ca-ee44d7fdf60a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:11.193626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7848d9-40a5-4eaf-81ca-ee44d7fdf60a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:11.193626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7848d9-40a5-4eaf-81ca-ee44d7fdf60a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:11.194968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 660/1302] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:11.200802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:11.201269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:11.201427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:11.219261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:11.221036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:11.294128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 643/1303] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:11.321403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7848d9-40a5-4eaf-81ca-ee44d7fdf60a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:11.424160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:11.431572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7848d9-40a5-4eaf-81ca-ee44d7fdf60a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:11.468878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:11.470426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:11.470997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:11.470997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:11.471650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:11.473126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:11.486213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:11.488165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 21 22:23:11.489870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 661/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jul 21 22:23:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:11.494008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:11.583012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:11.607658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe76f3-73ac-4569-8eda-2fa4d390d6d3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:11.646339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:11.647144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:11.647419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:11.648371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:11.649151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:11.649521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:11.650911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe9ad4-0831-4c2b-8ad5-ca1e78219701 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:11.656903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 644/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:11.664023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:11.773923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6079a-da82-4f56-af0e-e3f1908b5308 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:11.811270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3947ba2-b89d-48d4-aceb-52f88c8b8fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:11.812005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947ba2-b89d-48d4-aceb-52f88c8b8fc1 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:11.812288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947ba2-b89d-48d4-aceb-52f88c8b8fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:11.812627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947ba2-b89d-48d4-aceb-52f88c8b8fc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:11.814070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947ba2-b89d-48d4-aceb-52f88c8b8fc1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:11.815492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947ba2-b89d-48d4-aceb-52f88c8b8fc1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:11.835488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 662/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jul 21 22:23:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:23:21.653825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=30 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:21.830942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:19.681238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd88eaf-ab46-4527-b517-b0a43b635f46 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:19.681816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd88eaf-ab46-4527-b517-b0a43b635f46 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:19.681816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd88eaf-ab46-4527-b517-b0a43b635f46 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:19.687884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 645/1307] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:19.689606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1d227-a91f-4296-8381-0c224281659a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:19.690081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1d227-a91f-4296-8381-0c224281659a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:19.690167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1d227-a91f-4296-8381-0c224281659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:19.793209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e1c1d227-a91f-4296-8381-0c224281659a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:19.795498 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e1c1d227-a91f-4296-8381-0c224281659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:19.946209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 663/1308] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:20.011803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b9de5-c8e8-4732-9fba-e73cc2fb238c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:20.012074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b9de5-c8e8-4732-9fba-e73cc2fb238c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:20.012347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b9de5-c8e8-4732-9fba-e73cc2fb238c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:20.015583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 646/1309] 10.222.0.58 () {54 vars in 965 bytes} [Thu Jul 21 22:24:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:20.022118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dc265-fe45-41db-8e09-696d11de68b3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:20.022396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dc265-fe45-41db-8e09-696d11de68b3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:20.022681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dc265-fe45-41db-8e09-696d11de68b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:20.157601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-de2dc265-fe45-41db-8e09-696d11de68b3 None None] MFA Rules not processed for user `7f5dceea71d1434796a105ea67773cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:20.158927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-de2dc265-fe45-41db-8e09-696d11de68b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:20.187684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:20.258581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 664/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Thu Jul 21 22:24:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:24:20.269350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b9de5-c8e8-4732-9fba-e73cc2fb238c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:20.271526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bc0b9de5-c8e8-4732-9fba-e73cc2fb238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:20.403322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b9de5-c8e8-4732-9fba-e73cc2fb238c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:20.488136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:20.488717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:20.488992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:20.489262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:20.490967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:20.553801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:20.624964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:20.626453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d935973-47c2-4074-9b30-00d1f71f85d9 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:20.651747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 647/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:23.139454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f359dbd-bedf-4670-a9bb-063de71b63d6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:23.140060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f359dbd-bedf-4670-a9bb-063de71b63d6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:23.140060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f359dbd-bedf-4670-a9bb-063de71b63d6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:23.141442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 665/1312] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:23.147000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956f50d-cf82-49a9-9fa4-223eafb5fd35 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:23.147567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956f50d-cf82-49a9-9fa4-223eafb5fd35 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:23.147665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956f50d-cf82-49a9-9fa4-223eafb5fd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:23.243219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f956f50d-cf82-49a9-9fa4-223eafb5fd35 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:23.244482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f956f50d-cf82-49a9-9fa4-223eafb5fd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:23.354577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 648/1313] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:23.394557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f359dbd-bedf-4670-a9bb-063de71b63d6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:23.396294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7f359dbd-bedf-4670-a9bb-063de71b63d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:23.501207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f359dbd-bedf-4670-a9bb-063de71b63d6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:23.580061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:23.587928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:23.588589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:23.588872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:23.589151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:23.590826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:23.719643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:23.721110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d90a8a-8e1e-40f0-a539-f40ddb0cd274 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:23.739385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 666/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:26.209831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1991fa-00e7-4422-a008-63910bce9056 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:26.210412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1991fa-00e7-4422-a008-63910bce9056 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:26.210412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1991fa-00e7-4422-a008-63910bce9056 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:26.211830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 649/1315] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:26.217771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82220-0468-4957-82bd-486cad8bbd20 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:26.218235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82220-0468-4957-82bd-486cad8bbd20 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:26.218346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82220-0468-4957-82bd-486cad8bbd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:26.287475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a5d82220-0468-4957-82bd-486cad8bbd20 None None] MFA Rules not processed for user `a04367f4bff64bab991a70202530d2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:26.288706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5d82220-0468-4957-82bd-486cad8bbd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:26.341040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:26.391377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 667/1316] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:24:26.433979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1991fa-00e7-4422-a008-63910bce9056 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:26.435635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9b1991fa-00e7-4422-a008-63910bce9056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:26.445697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:26.541863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1991fa-00e7-4422-a008-63910bce9056 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:26.617608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:26.618361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:26.618643 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:26.618927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:26.620603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:26.695232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:26.747191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:26.748830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdd9f1-1fbb-40de-a331-0e03bd678e95 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:26.768388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 650/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:29.521904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0561dba-0e70-4f2a-887c-76fb1b286dba None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:29.522486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0561dba-0e70-4f2a-887c-76fb1b286dba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:29.522486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0561dba-0e70-4f2a-887c-76fb1b286dba None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:29.523866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 668/1318] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:29.529732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708f2d5-ad15-4abf-b8ab-08e1e8d4d570 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:29.530183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708f2d5-ad15-4abf-b8ab-08e1e8d4d570 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:29.530404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708f2d5-ad15-4abf-b8ab-08e1e8d4d570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:29.593697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6708f2d5-ad15-4abf-b8ab-08e1e8d4d570 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:29.594897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6708f2d5-ad15-4abf-b8ab-08e1e8d4d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:29.697579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 651/1319] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:29.755325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cbce8-e720-41cb-aa92-470e25a9ad2d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:29.755612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cbce8-e720-41cb-aa92-470e25a9ad2d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:29.755913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cbce8-e720-41cb-aa92-470e25a9ad2d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:29.757896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 669/1320] 10.222.0.58 () {54 vars in 965 bytes} [Thu Jul 21 22:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:29.765885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df8160-dd2b-457f-86f1-9577454c50b9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:29.766175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df8160-dd2b-457f-86f1-9577454c50b9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:29.766504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df8160-dd2b-457f-86f1-9577454c50b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:29.790666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:29.850391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-85df8160-dd2b-457f-86f1-9577454c50b9 None None] MFA Rules not processed for user `7f5dceea71d1434796a105ea67773cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:29.851671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-85df8160-dd2b-457f-86f1-9577454c50b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:29.938353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 652/1321] 10.222.0.58 () {58 vars in 1062 bytes} [Thu Jul 21 22:24:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:29.950111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cbce8-e720-41cb-aa92-470e25a9ad2d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:29.952421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ff5cbce8-e720-41cb-aa92-470e25a9ad2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:30.062873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cbce8-e720-41cb-aa92-470e25a9ad2d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:30.143074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:30.143750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:30.144036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:30.144313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:30.145985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:30.154395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:30.267491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:30.269478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db43bab-407a-4073-b263-8c62c0f6b437 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:30.290286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 670/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:33.592886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c3ee-55d3-4f9a-9d66-abc65c303980 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:33.593491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c3ee-55d3-4f9a-9d66-abc65c303980 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:33.593491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c3ee-55d3-4f9a-9d66-abc65c303980 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:33.594938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 653/1323] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:33.601083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65e7a1-3a9c-46a0-a7a2-095772f71aab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:33.601633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65e7a1-3a9c-46a0-a7a2-095772f71aab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:33.601633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65e7a1-3a9c-46a0-a7a2-095772f71aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:33.670864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9e65e7a1-3a9c-46a0-a7a2-095772f71aab None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:33.672128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9e65e7a1-3a9c-46a0-a7a2-095772f71aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:33.742171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:33.787246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 671/1324] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:33.824285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c3ee-55d3-4f9a-9d66-abc65c303980 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:33.826466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f326c3ee-55d3-4f9a-9d66-abc65c303980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:33.941356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c3ee-55d3-4f9a-9d66-abc65c303980 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:33.954199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:34.019698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:34.020358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:34.020641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:34.020916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:34.022619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:34.132353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:34.133685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848dc50-3c07-4e0d-a097-ae8d8e666f9e service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:34.156676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 654/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:36.269882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2ea44-5241-4e4a-bdd1-3cd4e86eb654 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:36.270407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2ea44-5241-4e4a-bdd1-3cd4e86eb654 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:36.270407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2ea44-5241-4e4a-bdd1-3cd4e86eb654 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:36.272147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 672/1326] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:36.278185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bcdaa-b99d-4847-993a-461868d11e0a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:36.278666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bcdaa-b99d-4847-993a-461868d11e0a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:36.278796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bcdaa-b99d-4847-993a-461868d11e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:36.330967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:36.351350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8a5bcdaa-b99d-4847-993a-461868d11e0a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:36.352575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a5bcdaa-b99d-4847-993a-461868d11e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:36.480537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 655/1327] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:36.521443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2ea44-5241-4e4a-bdd1-3cd4e86eb654 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:36.523854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-53d2ea44-5241-4e4a-bdd1-3cd4e86eb654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:36.642065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2ea44-5241-4e4a-bdd1-3cd4e86eb654 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:36.733979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:36.734634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:36.734914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:36.735261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:36.736892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:36.755388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:36.857043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:36.858576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0fb93-ad7a-4290-ba60-cebbd6d61c4f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:36.888223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 673/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:39.066984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82beec-0445-4123-b106-4cb08cf36769 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:39.067603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82beec-0445-4123-b106-4cb08cf36769 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:39.067603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82beec-0445-4123-b106-4cb08cf36769 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:39.068877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 656/1329] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:39.074655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7404-896a-458e-b156-5df67fcf40cb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:39.075463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7404-896a-458e-b156-5df67fcf40cb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:39.075463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7404-896a-458e-b156-5df67fcf40cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:39.150149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bccc7404-896a-458e-b156-5df67fcf40cb None None] MFA Rules not processed for user `a04367f4bff64bab991a70202530d2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:39.151414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bccc7404-896a-458e-b156-5df67fcf40cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:39.225395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:39.261510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 674/1330] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:24:39.300967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82beec-0445-4123-b106-4cb08cf36769 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:39.302530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5f82beec-0445-4123-b106-4cb08cf36769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:39.415134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82beec-0445-4123-b106-4cb08cf36769 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:39.437217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:39.494613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:39.495311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:39.495589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:39.495865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:39.497526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:39.630140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:39.631688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74552a3-4976-4e68-b209-b5b369fbbd92 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:39.652220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 657/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:24:42.032765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d701d1-6328-4d38-aba5-f3486674f7d4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:42.033301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d701d1-6328-4d38-aba5-f3486674f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:42.033301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d701d1-6328-4d38-aba5-f3486674f7d4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:42.034637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 675/1332] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:42.042807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb6f77-66d7-453e-8185-079e691c18e6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:42.043320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb6f77-66d7-453e-8185-079e691c18e6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:42.043416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb6f77-66d7-453e-8185-079e691c18e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:42.113719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:42.116698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e3cb6f77-66d7-453e-8185-079e691c18e6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:42.118610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e3cb6f77-66d7-453e-8185-079e691c18e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:42.233264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 658/1333] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:42.271315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d701d1-6328-4d38-aba5-f3486674f7d4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:42.272875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e4d701d1-6328-4d38-aba5-f3486674f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:42.381710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d701d1-6328-4d38-aba5-f3486674f7d4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:42.461989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:42.462637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:42.462922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:42.463226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:42.464905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:42.493451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:42.585399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:42.586863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9e1e8-31a2-49f1-a3c3-da778c21e593 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:42.605362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 676/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:44.726926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5154e-5970-4867-8a0d-e5796aafecb9 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:44.727495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5154e-5970-4867-8a0d-e5796aafecb9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:44.727495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5154e-5970-4867-8a0d-e5796aafecb9 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:44.728901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 659/1335] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:44.734881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427573-3e6e-49ef-b475-a0293d094787 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:44.735383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427573-3e6e-49ef-b475-a0293d094787 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:44.735568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427573-3e6e-49ef-b475-a0293d094787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:44.813060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fd427573-3e6e-49ef-b475-a0293d094787 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:44.814462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fd427573-3e6e-49ef-b475-a0293d094787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:44.894993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:44.919920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 677/1336] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:24:44.969479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5154e-5970-4867-8a0d-e5796aafecb9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:44.971437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-17f5154e-5970-4867-8a0d-e5796aafecb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:45.093747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5154e-5970-4867-8a0d-e5796aafecb9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:45.125666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:45.181291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:45.181863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:45.182150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:45.182456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:45.184272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:45.306808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:45.308348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e272f45-ad0e-4839-b179-629bd0d0f712 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:45.334623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 660/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:48.053348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac583c-61b5-48e5-93f9-877ec7b058ce None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:48.053877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac583c-61b5-48e5-93f9-877ec7b058ce None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:48.053877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac583c-61b5-48e5-93f9-877ec7b058ce None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:48.055222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 678/1338] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:24:48.061643 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4e87-e9e0-4cbb-845a-4bd4c1aaa46a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:48.062222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4e87-e9e0-4cbb-845a-4bd4c1aaa46a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:48.062222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4e87-e9e0-4cbb-845a-4bd4c1aaa46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:48.135459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bedf4e87-e9e0-4cbb-845a-4bd4c1aaa46a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:48.136719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bedf4e87-e9e0-4cbb-845a-4bd4c1aaa46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:48.146958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:48.254640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 661/1339] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:48.298328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac583c-61b5-48e5-93f9-877ec7b058ce None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:48.301299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-12ac583c-61b5-48e5-93f9-877ec7b058ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:48.443652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac583c-61b5-48e5-93f9-877ec7b058ce None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:48.525326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:48.526056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:48.526469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:48.526794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:48.528604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:48.570644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:48.657688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:48.660161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf4fb-dc57-43fd-a20d-b2aa506c2c93 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:48.683534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 679/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:53.221165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6eadad-473d-4564-881a-9bdb44a0a756 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:53.222133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6eadad-473d-4564-881a-9bdb44a0a756 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:53.222133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6eadad-473d-4564-881a-9bdb44a0a756 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:53.224667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 662/1341] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:53.235303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded863e9-6b09-4c27-af58-42f74e6bffe3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:53.236058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded863e9-6b09-4c27-af58-42f74e6bffe3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:53.236247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded863e9-6b09-4c27-af58-42f74e6bffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:53.347642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ded863e9-6b09-4c27-af58-42f74e6bffe3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:53.349822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ded863e9-6b09-4c27-af58-42f74e6bffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:53.494370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:53.523134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 680/1342] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:53.572344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6eadad-473d-4564-881a-9bdb44a0a756 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:53.574008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ac6eadad-473d-4564-881a-9bdb44a0a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:53.751760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6eadad-473d-4564-881a-9bdb44a0a756 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:53.824033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:53.869019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:53.869583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:53.869864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:53.870148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:53.872037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:54.045817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:54.047440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4f860-b9e5-4bca-a743-8a583c8b5e43 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:54.072336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 663/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:24:56.957350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d895ab-b725-4c21-95cd-3d5bd20dd26d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:56.957886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d895ab-b725-4c21-95cd-3d5bd20dd26d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:56.957886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d895ab-b725-4c21-95cd-3d5bd20dd26d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:56.959547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 681/1344] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:56.964644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331be23-3657-4589-afc0-1c67332ecaba None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:56.965138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331be23-3657-4589-afc0-1c67332ecaba None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:56.965261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331be23-3657-4589-afc0-1c67332ecaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:57.044400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d331be23-3657-4589-afc0-1c67332ecaba None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:57.046037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d331be23-3657-4589-afc0-1c67332ecaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:57.069740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:57.169528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 664/1345] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:57.210816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d895ab-b725-4c21-95cd-3d5bd20dd26d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:57.212343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c4d895ab-b725-4c21-95cd-3d5bd20dd26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:57.333240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d895ab-b725-4c21-95cd-3d5bd20dd26d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:57.412919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:57.413579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:57.413862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:57.414135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:57.415839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:57.463485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:57.534571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:57.536068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d12bc-a9e9-4bb5-9c70-4866bd78b061 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:57.556103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 682/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:02.098751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0c5d-aae7-416e-ae21-cc76922770fc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:02.098751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0c5d-aae7-416e-ae21-cc76922770fc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:02.098751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0c5d-aae7-416e-ae21-cc76922770fc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:02.110872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6320905-981f-4475-80ad-9ce75c8ecb5a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:02.110872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6320905-981f-4475-80ad-9ce75c8ecb5a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:02.110872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6320905-981f-4475-80ad-9ce75c8ecb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:02.112381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 665/1347] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:02.213629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a6320905-981f-4475-80ad-9ce75c8ecb5a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:02.215484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a6320905-981f-4475-80ad-9ce75c8ecb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:02.375452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:02.387133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 683/1348] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:02.423443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0c5d-aae7-416e-ae21-cc76922770fc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:02.428680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c40d0c5d-aae7-416e-ae21-cc76922770fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:02.554007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0c5d-aae7-416e-ae21-cc76922770fc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:02.610302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:02.644861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:02.645528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:02.645923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:02.646200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:02.647987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:02.781102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:02.782561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b2786-06e3-469e-8145-8e66d48dcf1e service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:02.801677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 666/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:08.264768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990c91-f0a0-4826-8478-da244a17e779 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:08.265347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990c91-f0a0-4826-8478-da244a17e779 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:08.265569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba990c91-f0a0-4826-8478-da244a17e779 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:08.268331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 684/1350] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:08.274447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60440c15-9c2c-4af4-83ac-2c4086226999 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:08.274951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60440c15-9c2c-4af4-83ac-2c4086226999 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:08.275084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60440c15-9c2c-4af4-83ac-2c4086226999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:08.398459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-60440c15-9c2c-4af4-83ac-2c4086226999 None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:08.400110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-60440c15-9c2c-4af4-83ac-2c4086226999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:08.425274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:08.564367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 667/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:25:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:13.124587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1be7f3-996e-4c1b-b0d4-77562dea07ee None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:13.125605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1be7f3-996e-4c1b-b0d4-77562dea07ee None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:13.126953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1be7f3-996e-4c1b-b0d4-77562dea07ee None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:13.129191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 685/1352] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:25:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:13.137461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb1a9-f9ce-4bb9-b231-2180c177270c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:13.139102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb1a9-f9ce-4bb9-b231-2180c177270c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:13.139397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb1a9-f9ce-4bb9-b231-2180c177270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:13.236001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1c0bb1a9-f9ce-4bb9-b231-2180c177270c None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:13.238154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1c0bb1a9-f9ce-4bb9-b231-2180c177270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:13.349483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:13.418631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 668/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:25:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:14.517514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c19c-3943-47c6-b5ed-2a71a1eb0379 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:14.518320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c19c-3943-47c6-b5ed-2a71a1eb0379 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:14.518320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c19c-3943-47c6-b5ed-2a71a1eb0379 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:14.519621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 686/1354] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:14.525185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec287a-4a19-453e-9f5f-3dd9115cab31 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:14.525757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec287a-4a19-453e-9f5f-3dd9115cab31 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:14.525757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec287a-4a19-453e-9f5f-3dd9115cab31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:14.619962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-beec287a-4a19-453e-9f5f-3dd9115cab31 None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:14.621195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-beec287a-4a19-453e-9f5f-3dd9115cab31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:14.779538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 669/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:25:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:18.513041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8393618-4c1c-4594-bdad-79eda6ee8946 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.513545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8393618-4c1c-4594-bdad-79eda6ee8946 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.513545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8393618-4c1c-4594-bdad-79eda6ee8946 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.519495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 687/1356] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:18.521912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841321e9-e965-4e1d-9ab3-2d4e5cbe1cf8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.522372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841321e9-e965-4e1d-9ab3-2d4e5cbe1cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.523193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841321e9-e965-4e1d-9ab3-2d4e5cbe1cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.567070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:18.609351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-841321e9-e965-4e1d-9ab3-2d4e5cbe1cf8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:18.610835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-841321e9-e965-4e1d-9ab3-2d4e5cbe1cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:18.739222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 670/1357] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:18.797200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2519894-0bf0-49c7-b410-c95ad24e37a1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.797492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2519894-0bf0-49c7-b410-c95ad24e37a1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.797849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2519894-0bf0-49c7-b410-c95ad24e37a1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.799169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 688/1358] 10.222.0.58 () {54 vars in 962 bytes} [Thu Jul 21 22:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:18.810528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbd9a6-093b-4317-b100-f1aee5cc93c3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.810813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbd9a6-093b-4317-b100-f1aee5cc93c3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.811134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbd9a6-093b-4317-b100-f1aee5cc93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.925637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-20bbd9a6-093b-4317-b100-f1aee5cc93c3 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:18.928149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-20bbd9a6-093b-4317-b100-f1aee5cc93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:18.990408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:19.091509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 671/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Jul 21 22:25:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:19.101677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2519894-0bf0-49c7-b410-c95ad24e37a1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:19.103410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d2519894-0bf0-49c7-b410-c95ad24e37a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:19.290833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2519894-0bf0-49c7-b410-c95ad24e37a1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:19.396830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:19.447232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:19.447895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:19.448177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:19.448461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:19.450334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:19.552950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:19.554589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc43ab6-962d-403b-bc7d-92254e28a8ec service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:19.573754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 689/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:21.863892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491eb13-329b-48fd-bee9-00074e15d988 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:21.864445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491eb13-329b-48fd-bee9-00074e15d988 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:21.864445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491eb13-329b-48fd-bee9-00074e15d988 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:21.866458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 672/1361] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:21.872159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab364-9de8-4a1f-875d-38b2e8ed2ba6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:21.873185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab364-9de8-4a1f-875d-38b2e8ed2ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:21.873468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab364-9de8-4a1f-875d-38b2e8ed2ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:21.955376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9f6ab364-9de8-4a1f-875d-38b2e8ed2ba6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:21.956701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9f6ab364-9de8-4a1f-875d-38b2e8ed2ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:22.010808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:22.100583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 690/1362] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:22.148577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491eb13-329b-48fd-bee9-00074e15d988 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:22.150102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8491eb13-329b-48fd-bee9-00074e15d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:22.339471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491eb13-329b-48fd-bee9-00074e15d988 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:22.350689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:22.484981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:22.485633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:22.485919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:22.486219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:22.487954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:22.588310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:22.589861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952b8fd-ad5f-4fb4-9278-7ce33343fd24 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:22.612167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 673/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:25.149028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7705db-a06f-4d75-ad9f-e2a8fcd20d79 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:25.149626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7705db-a06f-4d75-ad9f-e2a8fcd20d79 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:25.149626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7705db-a06f-4d75-ad9f-e2a8fcd20d79 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:25.150959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 691/1364] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:25.157330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942165a7-4f2f-40f3-a6df-4d37e9fa4c09 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:25.157910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942165a7-4f2f-40f3-a6df-4d37e9fa4c09 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:25.157910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942165a7-4f2f-40f3-a6df-4d37e9fa4c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:25.173983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:25.234425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-942165a7-4f2f-40f3-a6df-4d37e9fa4c09 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:25.235924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-942165a7-4f2f-40f3-a6df-4d37e9fa4c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:25.355449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 674/1365] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:25.404312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418761-7cd0-4471-b3b8-85b9baaf029c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:25.404600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418761-7cd0-4471-b3b8-85b9baaf029c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:25.405668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418761-7cd0-4471-b3b8-85b9baaf029c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:25.407609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 692/1366] 10.222.0.58 () {54 vars in 962 bytes} [Thu Jul 21 22:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:25.414297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4ee2-1b44-48ff-a88b-ed7c16207673 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:25.414603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4ee2-1b44-48ff-a88b-ed7c16207673 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:25.414884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4ee2-1b44-48ff-a88b-ed7c16207673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:25.514479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3aef4ee2-1b44-48ff-a88b-ed7c16207673 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:25.516587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3aef4ee2-1b44-48ff-a88b-ed7c16207673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:25.559048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:25.681445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 675/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Jul 21 22:25:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:25.697259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418761-7cd0-4471-b3b8-85b9baaf029c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:25.702703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bc418761-7cd0-4471-b3b8-85b9baaf029c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:25.949303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418761-7cd0-4471-b3b8-85b9baaf029c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:25.991763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:26.172572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:26.173674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:26.174167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:26.174665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:26.177998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:26.323527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:26.326745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f28890-c546-489a-bb3c-e7f2d07da0e6 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:26.365190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 693/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:27.120346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e125ae-9413-4ddd-8536-f59c22db6081 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:27.120912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e125ae-9413-4ddd-8536-f59c22db6081 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:27.121430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e125ae-9413-4ddd-8536-f59c22db6081 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:27.123653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 676/1369] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:27.135245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd069dc1-877e-4aea-a693-f833b30141f3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:27.135603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd069dc1-877e-4aea-a693-f833b30141f3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:27.135902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd069dc1-877e-4aea-a693-f833b30141f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:27.180682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:27.223179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dd069dc1-877e-4aea-a693-f833b30141f3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:27.224836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dd069dc1-877e-4aea-a693-f833b30141f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:27.362447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 694/1370] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:27.756708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef531d88-2680-42d7-a808-3a1d90beb3fb None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:27.757094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef531d88-2680-42d7-a808-3a1d90beb3fb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:27.757443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef531d88-2680-42d7-a808-3a1d90beb3fb None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:27.759966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 677/1371] 10.222.0.58 () {54 vars in 964 bytes} [Thu Jul 21 22:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:27.766345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af353-b6db-4123-a1fe-a214305ec90a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:27.766761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af353-b6db-4123-a1fe-a214305ec90a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:27.767128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af353-b6db-4123-a1fe-a214305ec90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:27.890777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f86af353-b6db-4123-a1fe-a214305ec90a None None] MFA Rules not processed for user `d529f6af558a44ef909937d5a960cb88`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:27.892549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f86af353-b6db-4123-a1fe-a214305ec90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:27.958546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:28.012121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 695/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Jul 21 22:25:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:28.021506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef531d88-2680-42d7-a808-3a1d90beb3fb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:28.024498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ef531d88-2680-42d7-a808-3a1d90beb3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:28.211161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef531d88-2680-42d7-a808-3a1d90beb3fb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:28.286451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:28.355813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:28.356257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:28.356752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:28.357064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:28.360693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:28.551979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:28.554197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dea11-4f8d-4f60-bb3b-5a0459a0cad2 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:28.586735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 678/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:28.776821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f006900-ad4b-4c8f-9669-0e3633f11c7f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:28.777343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f006900-ad4b-4c8f-9669-0e3633f11c7f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:28.777343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f006900-ad4b-4c8f-9669-0e3633f11c7f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:28.778634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 696/1374] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:28.784324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd3cd-a5a2-44c1-9e1a-3a1edaf8e156 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:28.784612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd3cd-a5a2-44c1-9e1a-3a1edaf8e156 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:28.784893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd3cd-a5a2-44c1-9e1a-3a1edaf8e156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:28.851252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6a0fd3cd-a5a2-44c1-9e1a-3a1edaf8e156 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:28.852573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6a0fd3cd-a5a2-44c1-9e1a-3a1edaf8e156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:28.874585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:28.960470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 679/1375] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:29.009748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f006900-ad4b-4c8f-9669-0e3633f11c7f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:29.011560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4f006900-ad4b-4c8f-9669-0e3633f11c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:29.254399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f006900-ad4b-4c8f-9669-0e3633f11c7f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:29.319263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:29.383970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:29.384680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:29.384971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:29.385255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:29.387025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:29.486396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:29.488007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddebb96-db13-412a-91db-e1e4e353b788 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:29.506907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 697/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:31.663391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95f3c-7852-498a-9fef-acbc7f872bf8 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:31.663814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95f3c-7852-498a-9fef-acbc7f872bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:31.663952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95f3c-7852-498a-9fef-acbc7f872bf8 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:31.665350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 680/1377] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:31.671741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4814e-2087-433e-9315-4cca01f43295 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:31.672206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4814e-2087-433e-9315-4cca01f43295 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:31.672206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4814e-2087-433e-9315-4cca01f43295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:31.770780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c6b4814e-2087-433e-9315-4cca01f43295 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:31.772049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c6b4814e-2087-433e-9315-4cca01f43295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:31.783208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:31.919879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 698/1378] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:31.967360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95f3c-7852-498a-9fef-acbc7f872bf8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:31.970265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0db95f3c-7852-498a-9fef-acbc7f872bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:32.190044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95f3c-7852-498a-9fef-acbc7f872bf8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:32.194618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:32.359779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:32.360880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:32.361351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:32.361807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:32.363566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:32.488743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:32.490455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad093e1-c304-4728-b554-3aa0ea76df48 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:32.514351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 681/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:34.692401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e98fa-d7f2-4059-ad63-637e0682c27e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:34.693093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e98fa-d7f2-4059-ad63-637e0682c27e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:34.693093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e98fa-d7f2-4059-ad63-637e0682c27e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:34.694973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 699/1380] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:34.701075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6b967-bf30-44e3-b296-2485526fd03b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:34.701596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6b967-bf30-44e3-b296-2485526fd03b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:34.701689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6b967-bf30-44e3-b296-2485526fd03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:34.713967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:34.804517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9fe6b967-bf30-44e3-b296-2485526fd03b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:34.806412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9fe6b967-bf30-44e3-b296-2485526fd03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:34.969689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 682/1381] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:35.025171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e98fa-d7f2-4059-ad63-637e0682c27e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:35.028390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fd3e98fa-d7f2-4059-ad63-637e0682c27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:35.225964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:35.242412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e98fa-d7f2-4059-ad63-637e0682c27e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:35.412818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:35.414098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:35.414693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:35.415341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:35.418278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:35.553490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:35.554287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5b05c-c57b-4d3c-a173-4cb8f5086171 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:35.567447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:35.578896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 700/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:37.978170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5de18-7715-4488-bf83-ae750f427919 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:37.978938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5de18-7715-4488-bf83-ae750f427919 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:37.978938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5de18-7715-4488-bf83-ae750f427919 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:37.980043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 683/1383] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:37.986770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528180e-0660-4b1e-a2d5-41db74ef8d12 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:37.987279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528180e-0660-4b1e-a2d5-41db74ef8d12 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:37.987419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528180e-0660-4b1e-a2d5-41db74ef8d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:38.054161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6528180e-0660-4b1e-a2d5-41db74ef8d12 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:38.055423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6528180e-0660-4b1e-a2d5-41db74ef8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:38.220697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 701/1384] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:38.266478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5de18-7715-4488-bf83-ae750f427919 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:38.268333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-42c5de18-7715-4488-bf83-ae750f427919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:38.389723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:38.477183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5de18-7715-4488-bf83-ae750f427919 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:38.627612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:38.628277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:38.628566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:38.628835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:38.630581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:38.702652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:38.753842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:38.755534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c53fb5-371a-4aa4-9d07-e7bedd05335f service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:38.791263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 684/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:41.944720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6168af-1670-4d23-80c8-5038f074cfc5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:41.945529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6168af-1670-4d23-80c8-5038f074cfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:41.945529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6168af-1670-4d23-80c8-5038f074cfc5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:41.947454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 702/1386] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:41.954282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087567d0-e0cd-4500-b9f3-2f6abf6e2bc4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:41.955119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087567d0-e0cd-4500-b9f3-2f6abf6e2bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:41.955119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087567d0-e0cd-4500-b9f3-2f6abf6e2bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:42.033041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-087567d0-e0cd-4500-b9f3-2f6abf6e2bc4 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:42.034503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-087567d0-e0cd-4500-b9f3-2f6abf6e2bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:42.149555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 685/1387] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:42.195907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6168af-1670-4d23-80c8-5038f074cfc5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:42.197512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0a6168af-1670-4d23-80c8-5038f074cfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:42.283506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:42.384038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6168af-1670-4d23-80c8-5038f074cfc5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:42.575173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:42.576281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:42.576741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:42.578203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:42.581249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:42.659892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:42.741478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:42.744422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c081e-4958-4d91-9325-8f2ed75c2e0c service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:42.782549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 703/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:44.573748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc932ac-4cb5-4004-afa7-07886eb0ae16 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:44.574553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc932ac-4cb5-4004-afa7-07886eb0ae16 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:44.574553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc932ac-4cb5-4004-afa7-07886eb0ae16 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:44.575665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 686/1389] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:44.583286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e9638-09c0-4660-bd7c-de1bb84f9803 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:44.584109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e9638-09c0-4660-bd7c-de1bb84f9803 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:44.584109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e9638-09c0-4660-bd7c-de1bb84f9803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:44.666632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2b9e9638-09c0-4660-bd7c-de1bb84f9803 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:44.668850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2b9e9638-09c0-4660-bd7c-de1bb84f9803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:44.815117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 704/1390] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:44.917709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa486a06-a81c-4d39-af3b-c73b9cb7601c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:44.918155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa486a06-a81c-4d39-af3b-c73b9cb7601c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:44.918588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa486a06-a81c-4d39-af3b-c73b9cb7601c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:44.921492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 687/1391] 10.222.0.58 () {54 vars in 966 bytes} [Thu Jul 21 22:25:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:44.930144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:44.930581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:44.931034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:44.959151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:45.055363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7795e-3855-4262-a9f1-4498a25e85cc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:45.055758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7795e-3855-4262-a9f1-4498a25e85cc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:45.055932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7795e-3855-4262-a9f1-4498a25e85cc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:45.057611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 688/1392] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:45.064439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:45.066041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:45.066739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:45.080424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] MFA Rules not processed for user `3df4115fcce342f2ae095ec1a2af4cca`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:45.082241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:45.112922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:45.156934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:45.158356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:45.196336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 705/1393] 10.222.0.58 () {58 vars in 1063 bytes} [Thu Jul 21 22:25:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:45.206993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:45.222790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190193c-3d6c-4dfb-a1ed-72b9507c1ce6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:45.282853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 689/1394] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:45.332215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:45.334161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:45.373459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:45.375878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:45.376490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:45.377003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:45.380162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:45.435671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:45.461825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:45.471297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd178fe0-37ed-4fd5-93dc-0889e99b299a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:45.562982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:45.565910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59474ffe-6070-4da0-b320-345b33befb03 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:45.600755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 706/1395] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:25:45.625065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:45.625856 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:45.626247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:45.626722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:45.629640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:45.749925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:45.752601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d55f62-bd04-4ea2-9445-ef4b6d6c4464 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:45.772368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:45.776850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 690/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:25:46.534523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85484ac-449f-40eb-8278-21c9235b66d9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:46.534904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85484ac-449f-40eb-8278-21c9235b66d9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:46.535279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85484ac-449f-40eb-8278-21c9235b66d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:46.614190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c85484ac-449f-40eb-8278-21c9235b66d9 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:46.615455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c85484ac-449f-40eb-8278-21c9235b66d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:46.718800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:46.729040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 707/1397] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:25:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:46.783264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c081437-6b82-4081-86cf-c87d8bd2aa37 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:46.783833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c081437-6b82-4081-86cf-c87d8bd2aa37 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:46.783833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c081437-6b82-4081-86cf-c87d8bd2aa37 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:46.785249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 691/1398] 10.222.0.58 () {54 vars in 943 bytes} [Thu Jul 21 22:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:46.793128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb350d-5752-4df8-8af9-a4ad968464af None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:46.793626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb350d-5752-4df8-8af9-a4ad968464af None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:46.794159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb350d-5752-4df8-8af9-a4ad968464af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:46.892627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d8fb350d-5752-4df8-8af9-a4ad968464af None None] MFA Rules not processed for user `08ed51a3aab34ac0bf5791fa3307a79d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:46.894084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8fb350d-5752-4df8-8af9-a4ad968464af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:47.021162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 708/1399] 10.222.0.58 () {58 vars in 1040 bytes} [Thu Jul 21 22:25:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:47.036571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c081437-6b82-4081-86cf-c87d8bd2aa37 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:47.038366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9c081437-6b82-4081-86cf-c87d8bd2aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:47.263556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c081437-6b82-4081-86cf-c87d8bd2aa37 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:47.394933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:47.419644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:47.420595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:47.421057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:47.421970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:47.424723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:47.533408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:47.535022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36312f-6b06-4cef-95ff-a0bbf39cb49b service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:47.543457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 692/1400] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:25:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 22:25:48.080171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52bf47-c320-47b5-a3ee-b2b190712acc None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:48.080843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52bf47-c320-47b5-a3ee-b2b190712acc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:48.080843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52bf47-c320-47b5-a3ee-b2b190712acc None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:48.082999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 709/1401] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:25:48.089806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097668-d301-4c2d-a5e7-fd3d6ff833fe None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:48.090567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097668-d301-4c2d-a5e7-fd3d6ff833fe None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:48.090567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097668-d301-4c2d-a5e7-fd3d6ff833fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:48.180146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4a097668-d301-4c2d-a5e7-fd3d6ff833fe None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:48.181415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4a097668-d301-4c2d-a5e7-fd3d6ff833fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:48.296755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:48.350306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 693/1402] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:48.399386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52bf47-c320-47b5-a3ee-b2b190712acc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:48.401091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bf52bf47-c320-47b5-a3ee-b2b190712acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:48.474251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:48.594823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52bf47-c320-47b5-a3ee-b2b190712acc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:48.822601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:48.823594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:48.824040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:48.824439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:48.831098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:48.862305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:48.961476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:48.963133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9649fe8-6b0d-4cff-86c4-b01a7a92d84c service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:48.982404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 710/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:51.224352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034106-4b53-415e-ad3e-3d5f2a0253c7 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:51.224956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034106-4b53-415e-ad3e-3d5f2a0253c7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:51.224956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034106-4b53-415e-ad3e-3d5f2a0253c7 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:51.227527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 694/1404] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:51.237283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489b8f8-7833-42b9-a905-8e9281f0f985 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:51.238547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489b8f8-7833-42b9-a905-8e9281f0f985 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:51.239777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489b8f8-7833-42b9-a905-8e9281f0f985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:51.322530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d489b8f8-7833-42b9-a905-8e9281f0f985 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:51.323944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d489b8f8-7833-42b9-a905-8e9281f0f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:51.435161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:51.446974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 711/1405] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:51.496861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034106-4b53-415e-ad3e-3d5f2a0253c7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:51.498475 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2d034106-4b53-415e-ad3e-3d5f2a0253c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:51.708806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034106-4b53-415e-ad3e-3d5f2a0253c7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:51.799194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:51.886530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:51.887425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:51.887781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:51.888165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:51.890797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:52.017373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:52.019224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8108374-d7a6-4cce-b00f-464b7d2a0ddf service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:52.051382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 695/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:25:54.308327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e96b6-d368-4731-a99a-e7effa2c79d8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:54.309041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e96b6-d368-4731-a99a-e7effa2c79d8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:54.309041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e96b6-d368-4731-a99a-e7effa2c79d8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:54.310443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 712/1407] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:54.322054 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42cbd6-3c0c-459e-be36-5251a4c008db None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:54.322926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42cbd6-3c0c-459e-be36-5251a4c008db None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:54.323132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42cbd6-3c0c-459e-be36-5251a4c008db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:54.405495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-dd42cbd6-3c0c-459e-be36-5251a4c008db None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:54.406933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd42cbd6-3c0c-459e-be36-5251a4c008db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:54.424315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:54.530022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 696/1408] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:54.578728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e96b6-d368-4731-a99a-e7effa2c79d8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:54.580343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-079e96b6-d368-4731-a99a-e7effa2c79d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:54.769576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e96b6-d368-4731-a99a-e7effa2c79d8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:54.856049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:54.927262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:54.927911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:54.928202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:54.928481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:54.930449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:55.050643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:55.052591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0fb11-6171-4c48-bb70-926408421ea7 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:55.083202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 713/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:04.528046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=37 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:05.080640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=65 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:10.847451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254646a-6e91-40e1-9436-5d9443d97bac None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:10.848036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254646a-6e91-40e1-9436-5d9443d97bac None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:10.848036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254646a-6e91-40e1-9436-5d9443d97bac None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:10.849611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 697/1410] 10.222.0.58 () {54 vars in 918 bytes} [Thu Jul 21 22:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:10.855959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485d049-dcd6-484c-8176-aebf6cc7229f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:10.856424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485d049-dcd6-484c-8176-aebf6cc7229f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:10.856693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485d049-dcd6-484c-8176-aebf6cc7229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:10.945253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c485d049-dcd6-484c-8176-aebf6cc7229f None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:10.946411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c485d049-dcd6-484c-8176-aebf6cc7229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:11.070109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 714/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 21 22:26:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:11.085433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f5d6-796f-41b9-884c-32b40e805953 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:11.085823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f5d6-796f-41b9-884c-32b40e805953 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:11.086109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f5d6-796f-41b9-884c-32b40e805953 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:11.087585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 698/1412] 10.222.0.58 () {54 vars in 918 bytes} [Thu Jul 21 22:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:11.094004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a9c4a-8d04-4ed6-bafa-1b0a49fe78f0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:11.094795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a9c4a-8d04-4ed6-bafa-1b0a49fe78f0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:11.095098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a9c4a-8d04-4ed6-bafa-1b0a49fe78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:11.175140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:11.178134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9b4a9c4a-8d04-4ed6-bafa-1b0a49fe78f0 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:11.180033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b4a9c4a-8d04-4ed6-bafa-1b0a49fe78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:11.308103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 715/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 21 22:26:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:13.900877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd61f9-1389-4967-89b6-9df8036ba966 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:13.901449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd61f9-1389-4967-89b6-9df8036ba966 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:13.901449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd61f9-1389-4967-89b6-9df8036ba966 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:13.902821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:13.908874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44260f-2d41-4769-b039-ef5ab395f6e8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:13.909516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44260f-2d41-4769-b039-ef5ab395f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:13.909516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44260f-2d41-4769-b039-ef5ab395f6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:13.976907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5d44260f-2d41-4769-b039-ef5ab395f6e8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:13.977993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5d44260f-2d41-4769-b039-ef5ab395f6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:14.048894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:14.091272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:14.139427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd61f9-1389-4967-89b6-9df8036ba966 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:14.141511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f2fd61f9-1389-4967-89b6-9df8036ba966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:14.328999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd61f9-1389-4967-89b6-9df8036ba966 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:14.410261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:14.479374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:14.480436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:14.480946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:14.481340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:14.483381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:14.583308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:14.584833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cb384-6ffd-4df1-9532-9fb913b919cb service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:14.604142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 700/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:15.589283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24d8b1-f0aa-4ca9-9841-cab3d938b6d0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:15.589906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24d8b1-f0aa-4ca9-9841-cab3d938b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:15.589906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24d8b1-f0aa-4ca9-9841-cab3d938b6d0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:15.591530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:15.597953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21136ee-6c14-40e7-9fc7-50b685cda88e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:15.598547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21136ee-6c14-40e7-9fc7-50b685cda88e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:15.598547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21136ee-6c14-40e7-9fc7-50b685cda88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:15.693346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c21136ee-6c14-40e7-9fc7-50b685cda88e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:15.694610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c21136ee-6c14-40e7-9fc7-50b685cda88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:15.724654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:15.796923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:15.966531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24d8b1-f0aa-4ca9-9841-cab3d938b6d0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:15.968222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2c24d8b1-f0aa-4ca9-9841-cab3d938b6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:16.123559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24d8b1-f0aa-4ca9-9841-cab3d938b6d0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:16.168319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:16.251334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:16.252027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:16.252314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:16.252592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:16.254305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:16.355728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:16.357367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912c5dd-3329-44cd-b6d9-c52def6891e8 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:16.378077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 718/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:17.306407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33839c73-47be-4f0e-8e68-17ae7befedde None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:17.307008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33839c73-47be-4f0e-8e68-17ae7befedde None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:17.307008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33839c73-47be-4f0e-8e68-17ae7befedde None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:17.308373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:17.313593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22f0f1-535d-4f12-aeae-5c5fc2a6a47a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:17.314086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22f0f1-535d-4f12-aeae-5c5fc2a6a47a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:17.314194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22f0f1-535d-4f12-aeae-5c5fc2a6a47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:17.372671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:17.401581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2d22f0f1-535d-4f12-aeae-5c5fc2a6a47a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:17.402812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2d22f0f1-535d-4f12-aeae-5c5fc2a6a47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:17.506548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:17.661362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33839c73-47be-4f0e-8e68-17ae7befedde None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:17.663339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-33839c73-47be-4f0e-8e68-17ae7befedde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:17.829040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33839c73-47be-4f0e-8e68-17ae7befedde None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:17.842202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:17.949224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:17.949793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:17.950079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:17.950357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:17.952085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:18.047871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:18.049506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3320a-8111-40cc-bddf-0395187ca16a service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:18.070887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 703/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:18.972585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb54adc-56e4-4e6b-9a42-1be127cb6255 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.973184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb54adc-56e4-4e6b-9a42-1be127cb6255 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.973184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb54adc-56e4-4e6b-9a42-1be127cb6255 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:18.974671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:18.980210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ff347-9e62-4e43-87bd-ebe24be89f72 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.980740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ff347-9e62-4e43-87bd-ebe24be89f72 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.980740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ff347-9e62-4e43-87bd-ebe24be89f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:19.001538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:19.070552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ab5ff347-9e62-4e43-87bd-ebe24be89f72 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:19.071867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ab5ff347-9e62-4e43-87bd-ebe24be89f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:19.185654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:19.336939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb54adc-56e4-4e6b-9a42-1be127cb6255 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:19.338064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fcb54adc-56e4-4e6b-9a42-1be127cb6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:19.467996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:19.509628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb54adc-56e4-4e6b-9a42-1be127cb6255 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:19.629704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:19.630373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:19.630653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:19.630927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:19.632701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:19.719660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:19.737508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:19.739243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd921245-6a8e-48b7-af30-4626f0b36874 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:19.761632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 721/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:20.659692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6231-abfa-40ec-9567-425f6b2b2b62 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.660298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6231-abfa-40ec-9567-425f6b2b2b62 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.660298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6231-abfa-40ec-9567-425f6b2b2b62 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.661985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:20.667388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ce15f-d0ae-4c01-b943-67099a85cd0d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.668077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ce15f-d0ae-4c01-b943-67099a85cd0d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.668077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ce15f-d0ae-4c01-b943-67099a85cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.761167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ac7ce15f-d0ae-4c01-b943-67099a85cd0d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:20.762423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ac7ce15f-d0ae-4c01-b943-67099a85cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.877699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:21.034209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6231-abfa-40ec-9567-425f6b2b2b62 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:21.035761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-26ff6231-abfa-40ec-9567-425f6b2b2b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:21.143925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:21.224450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6231-abfa-40ec-9567-425f6b2b2b62 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:21.360899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:21.361577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:21.361863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:21.362143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:21.364001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:21.426925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:21.476965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:21.478631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a452e0e-1362-45c2-9587-c4bd83396aef service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:21.500190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 706/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:22.333829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedf14c-5e88-45d0-b39d-3e0c1325a7b8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:22.334643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedf14c-5e88-45d0-b39d-3e0c1325a7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:22.334643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedf14c-5e88-45d0-b39d-3e0c1325a7b8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:22.336260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:22.341532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312b2ef-916e-4948-8576-f94712607d2d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:22.342021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312b2ef-916e-4948-8576-f94712607d2d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:22.342178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312b2ef-916e-4948-8576-f94712607d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:22.434604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e312b2ef-916e-4948-8576-f94712607d2d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:22.435913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e312b2ef-916e-4948-8576-f94712607d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:22.540702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:22.706499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedf14c-5e88-45d0-b39d-3e0c1325a7b8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:22.708584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4aedf14c-5e88-45d0-b39d-3e0c1325a7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:22.776983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:22.881634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedf14c-5e88-45d0-b39d-3e0c1325a7b8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:23.003483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:23.004054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:23.004352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:23.004631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:23.006327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:23.013974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:23.108164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:23.110386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e85b-9926-4981-9150-fbf05013760f service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:23.129922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 724/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:23.981142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fb18f-06bd-4012-a72e-36f6bc50e5ca None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:23.981780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fb18f-06bd-4012-a72e-36f6bc50e5ca None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:23.981780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fb18f-06bd-4012-a72e-36f6bc50e5ca None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:23.983113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:23.989513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc19731-572c-40bc-8d44-f6f55b2764b0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:23.990172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc19731-572c-40bc-8d44-f6f55b2764b0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:23.990172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc19731-572c-40bc-8d44-f6f55b2764b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:24.084164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dbc19731-572c-40bc-8d44-f6f55b2764b0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:24.085435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dbc19731-572c-40bc-8d44-f6f55b2764b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:24.164978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:24.212681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:24.372781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fb18f-06bd-4012-a72e-36f6bc50e5ca None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:24.375344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ab1fb18f-06bd-4012-a72e-36f6bc50e5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:24.396673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:24.558014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fb18f-06bd-4012-a72e-36f6bc50e5ca None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:24.656462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:24.685375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:24.686031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:24.686315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:24.686591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:24.688493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:24.796146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:24.797967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d980dc7-84c5-42af-952d-c38b96c2811a service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:24.818338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 709/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:25.676061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35a83a-f7b4-4a53-86f6-848fe664bb5f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:25.676611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35a83a-f7b4-4a53-86f6-848fe664bb5f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:25.676611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35a83a-f7b4-4a53-86f6-848fe664bb5f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:25.678018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:25.683227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377b26a-2b37-4779-8dac-192cabbd1ffa None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:25.683706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377b26a-2b37-4779-8dac-192cabbd1ffa None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:25.683860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377b26a-2b37-4779-8dac-192cabbd1ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:25.783477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2377b26a-2b37-4779-8dac-192cabbd1ffa None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:25.784703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2377b26a-2b37-4779-8dac-192cabbd1ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:25.829427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:25.887578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:26.043880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35a83a-f7b4-4a53-86f6-848fe664bb5f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:26.045676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd35a83a-f7b4-4a53-86f6-848fe664bb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:26.239799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35a83a-f7b4-4a53-86f6-848fe664bb5f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:26.324181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:26.388263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:26.388916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:26.389198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:26.389471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:26.391985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:26.501623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:26.503308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c64c09-d2c1-4dfb-bdf7-179891bf391f service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:26.523469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 727/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:27.379348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55172d-a8d5-4275-ba49-26d21edf76da None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:27.379936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55172d-a8d5-4275-ba49-26d21edf76da None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:27.379936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55172d-a8d5-4275-ba49-26d21edf76da None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:27.381279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:27.387464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315dc6a-4149-41a6-9b7b-0500130d3d1d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:27.388030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315dc6a-4149-41a6-9b7b-0500130d3d1d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:27.388030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315dc6a-4149-41a6-9b7b-0500130d3d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:27.473058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9315dc6a-4149-41a6-9b7b-0500130d3d1d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:27.474304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9315dc6a-4149-41a6-9b7b-0500130d3d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:27.476560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:27.584030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:27.739220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55172d-a8d5-4275-ba49-26d21edf76da None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:27.740327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a55172d-a8d5-4275-ba49-26d21edf76da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:27.894143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55172d-a8d5-4275-ba49-26d21edf76da None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:27.922690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:28.017238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:28.017888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:28.018177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:28.018450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:28.020499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:28.116018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:28.117647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcb1c2-1139-4850-b55a-3bd638478814 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:28.138775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 712/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:30.268995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caa29c-ad02-4811-8455-b29f166f10df None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:30.269847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caa29c-ad02-4811-8455-b29f166f10df None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:30.269847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caa29c-ad02-4811-8455-b29f166f10df None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:30.271226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:30.278186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375fe98-8163-4456-ba3c-c5378cb7e9f1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:30.278829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375fe98-8163-4456-ba3c-c5378cb7e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:30.278829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375fe98-8163-4456-ba3c-c5378cb7e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:30.338004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:30.380805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7375fe98-8163-4456-ba3c-c5378cb7e9f1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:30.382521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7375fe98-8163-4456-ba3c-c5378cb7e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:30.508993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:30.568746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caa29c-ad02-4811-8455-b29f166f10df None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:30.570624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c3caa29c-ad02-4811-8455-b29f166f10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:30.716902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caa29c-ad02-4811-8455-b29f166f10df None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:30.753938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:30.798407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:30.799114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:30.799414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:30.799862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:30.801952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:30.923824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:30.925273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dd068-573a-451e-a027-7970df37ebb2 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:30.945703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 730/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:33.282850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f80ac-18e8-4759-ad1a-e5388b0f4e6e None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:33.283484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f80ac-18e8-4759-ad1a-e5388b0f4e6e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:33.283484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f80ac-18e8-4759-ad1a-e5388b0f4e6e None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:33.284891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:33.290838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d54ee-ae8f-485c-87a5-ab9aae6a692b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:33.291437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d54ee-ae8f-485c-87a5-ab9aae6a692b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:33.291437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d54ee-ae8f-485c-87a5-ab9aae6a692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:33.363664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-132d54ee-ae8f-485c-87a5-ab9aae6a692b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:33.364927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-132d54ee-ae8f-485c-87a5-ab9aae6a692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:33.391548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:33.502746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:33.543354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f80ac-18e8-4759-ad1a-e5388b0f4e6e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:33.544952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3a6f80ac-18e8-4759-ad1a-e5388b0f4e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:33.659652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f80ac-18e8-4759-ad1a-e5388b0f4e6e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:33.673185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:33.743121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:33.743341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:33.743623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:33.743898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:33.745613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:33.882739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:33.884219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf019e-9770-4b20-b743-2aeb1eba07d7 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:33.902768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 715/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:36.321393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4846b7-5b52-4222-a811-f7800bbc5a6f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:36.321951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4846b7-5b52-4222-a811-f7800bbc5a6f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:36.321951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4846b7-5b52-4222-a811-f7800bbc5a6f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:36.323350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:36.329465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e406-958c-443d-b4bb-b4f01c389a10 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:36.329938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e406-958c-443d-b4bb-b4f01c389a10 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:36.330055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e406-958c-443d-b4bb-b4f01c389a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:36.378355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:36.398864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1880e406-958c-443d-b4bb-b4f01c389a10 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:36.400156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1880e406-958c-443d-b4bb-b4f01c389a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:36.502178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:36.542056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4846b7-5b52-4222-a811-f7800bbc5a6f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:36.543669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ad4846b7-5b52-4222-a811-f7800bbc5a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:36.653405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4846b7-5b52-4222-a811-f7800bbc5a6f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:36.696978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:36.732511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:36.733165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:36.733446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:36.734370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:36.736473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:36.852615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:36.854071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a4b68-2419-4280-a02b-e787f808daf7 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:36.873135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 733/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:39.123985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e563a-0d7a-4001-ab53-c69053f8f7f3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:39.124750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e563a-0d7a-4001-ab53-c69053f8f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:39.124935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e563a-0d7a-4001-ab53-c69053f8f7f3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:39.126327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:39.132048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776ef4-1f87-41f0-acb6-5239ebb75c03 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:39.132571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776ef4-1f87-41f0-acb6-5239ebb75c03 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:39.132571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776ef4-1f87-41f0-acb6-5239ebb75c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:39.201209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a0776ef4-1f87-41f0-acb6-5239ebb75c03 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:39.204006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0776ef4-1f87-41f0-acb6-5239ebb75c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:39.227761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:39.431968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:39.471649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e563a-0d7a-4001-ab53-c69053f8f7f3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:39.473199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-600e563a-0d7a-4001-ab53-c69053f8f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:39.581892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e563a-0d7a-4001-ab53-c69053f8f7f3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:39.602370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:39.659547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:39.660205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:39.660489 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:39.660764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:39.662442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:39.783563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:39.785020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71cffa-83fa-4db2-bb2c-f441bc54b375 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:39.804174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 718/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:41.945880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f5bc1-3968-4473-b750-b51c247376d8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:41.946455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f5bc1-3968-4473-b750-b51c247376d8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:41.946455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f5bc1-3968-4473-b750-b51c247376d8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:41.947757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:41.953609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2287b-5bc4-4e2a-b66c-cf63ca942529 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:41.954076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2287b-5bc4-4e2a-b66c-cf63ca942529 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:41.954185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2287b-5bc4-4e2a-b66c-cf63ca942529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:42.016694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:42.019717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-37d2287b-5bc4-4e2a-b66c-cf63ca942529 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:42.021750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-37d2287b-5bc4-4e2a-b66c-cf63ca942529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:42.118639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:42.157996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f5bc1-3968-4473-b750-b51c247376d8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:42.159564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-623f5bc1-3968-4473-b750-b51c247376d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:42.263771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f5bc1-3968-4473-b750-b51c247376d8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:42.314174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:42.339206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:42.339889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:42.340171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:42.340446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:42.342112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:42.453979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:42.455508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f58894-15bc-41ec-9ae7-de8273e5830e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:42.474610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 736/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:44.636636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa920d9-530d-4d75-af9b-d482c105f447 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:44.637409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa920d9-530d-4d75-af9b-d482c105f447 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:44.637409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa920d9-530d-4d75-af9b-d482c105f447 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:44.638604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:44.645524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd1207-7dd6-419b-bd18-17d66e575e4d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:44.646079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd1207-7dd6-419b-bd18-17d66e575e4d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:44.646079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd1207-7dd6-419b-bd18-17d66e575e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:44.723481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d9dd1207-7dd6-419b-bd18-17d66e575e4d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:44.724751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d9dd1207-7dd6-419b-bd18-17d66e575e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:44.757957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:44.834042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:44.873749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa920d9-530d-4d75-af9b-d482c105f447 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:44.876432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cfa920d9-530d-4d75-af9b-d482c105f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:44.985977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa920d9-530d-4d75-af9b-d482c105f447 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:45.016256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:45.068765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:45.069423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:45.069709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:45.069989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:45.071709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:45.182347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:45.183839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec62cf6-cd81-4034-8bc9-4a30c49b2b0c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:45.202177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 721/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:47.325368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63416d71-1d1f-401c-9a07-8f0d474dd45b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:47.325950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63416d71-1d1f-401c-9a07-8f0d474dd45b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:47.325950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63416d71-1d1f-401c-9a07-8f0d474dd45b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:47.327329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:47.333232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6400c6-bc50-4b88-8fee-c30e1f08e5e0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:47.333738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6400c6-bc50-4b88-8fee-c30e1f08e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:47.333834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6400c6-bc50-4b88-8fee-c30e1f08e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:47.402733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8d6400c6-bc50-4b88-8fee-c30e1f08e5e0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:47.404010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8d6400c6-bc50-4b88-8fee-c30e1f08e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:47.414162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:47.523177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:47.568244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63416d71-1d1f-401c-9a07-8f0d474dd45b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:47.569809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-63416d71-1d1f-401c-9a07-8f0d474dd45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:47.681016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63416d71-1d1f-401c-9a07-8f0d474dd45b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:47.738001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:47.757212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:47.757918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:47.758200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:47.758475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:47.760184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:47.871485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:47.872942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37afc6-24fb-4c97-9a5c-6f026a09b62d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:47.893054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 739/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:50.024096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaabfa-64c6-4f0a-a3d0-1876a6eec2d4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:50.024651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaabfa-64c6-4f0a-a3d0-1876a6eec2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:50.024651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaabfa-64c6-4f0a-a3d0-1876a6eec2d4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:50.025946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:50.032627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40aabe-44c0-4dd8-9941-68bf3d78178c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:50.033322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40aabe-44c0-4dd8-9941-68bf3d78178c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:50.033322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40aabe-44c0-4dd8-9941-68bf3d78178c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:50.103781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-be40aabe-44c0-4dd8-9941-68bf3d78178c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:50.105019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-be40aabe-44c0-4dd8-9941-68bf3d78178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:50.146193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:50.208943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:50.248044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaabfa-64c6-4f0a-a3d0-1876a6eec2d4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:50.250368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c6eaabfa-64c6-4f0a-a3d0-1876a6eec2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:50.358153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaabfa-64c6-4f0a-a3d0-1876a6eec2d4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:50.397495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:50.438087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:50.438691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:50.439033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:50.439395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:50.441196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:50.564316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:50.565761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0653606-b0dc-4428-b537-24aabde722ff service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:50.584039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 724/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:52.727470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d65e67-eef3-4b5b-bc6e-613a7e2fe7a0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:52.728334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d65e67-eef3-4b5b-bc6e-613a7e2fe7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:52.728508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d65e67-eef3-4b5b-bc6e-613a7e2fe7a0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:52.732138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:52.739189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700805e0-e3e6-4e74-9f70-4dfcbeeb8de2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:52.739757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700805e0-e3e6-4e74-9f70-4dfcbeeb8de2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:52.739757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700805e0-e3e6-4e74-9f70-4dfcbeeb8de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:52.821408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-700805e0-e3e6-4e74-9f70-4dfcbeeb8de2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:52.822663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-700805e0-e3e6-4e74-9f70-4dfcbeeb8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:52.840701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:52.924329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:52.960678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d65e67-eef3-4b5b-bc6e-613a7e2fe7a0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:52.962228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e6d65e67-eef3-4b5b-bc6e-613a7e2fe7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:53.067237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d65e67-eef3-4b5b-bc6e-613a7e2fe7a0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:53.134391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:53.142071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:53.142726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:53.143020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:53.143324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:53.145048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:53.257734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:53.259159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4638d-03fc-4d3e-83f3-fccb7cceae6b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:53.280068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 742/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:55.450568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00f670-befc-4fcf-a0ab-9be3f4e8febb None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:55.451227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00f670-befc-4fcf-a0ab-9be3f4e8febb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:55.451227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00f670-befc-4fcf-a0ab-9be3f4e8febb None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:55.452567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:55.458969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6eda8-6aad-4998-b154-8371c95f3d0f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:55.459478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6eda8-6aad-4998-b154-8371c95f3d0f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:55.459594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6eda8-6aad-4998-b154-8371c95f3d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:55.531395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-13a6eda8-6aad-4998-b154-8371c95f3d0f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:55.532895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-13a6eda8-6aad-4998-b154-8371c95f3d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:55.589304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:55.639401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:55.678049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00f670-befc-4fcf-a0ab-9be3f4e8febb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:55.679631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7e00f670-befc-4fcf-a0ab-9be3f4e8febb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:55.784853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00f670-befc-4fcf-a0ab-9be3f4e8febb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:55.833275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:55.860468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:55.861123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:55.861412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:55.861694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:55.863399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:55.975588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:55.977042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fb3ee-d627-45ac-9305-fa59ce0de650 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:55.995295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 727/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:05.637589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=22 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:05.993022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=66 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:16.146081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9269-4e6a-4949-8a5d-38f618d39552 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.146677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9269-4e6a-4949-8a5d-38f618d39552 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.146677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc9269-4e6a-4949-8a5d-38f618d39552 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.148084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 744/1471] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:16.155165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa231d1-ba7c-436b-8578-6f625401d1b4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.156200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa231d1-ba7c-436b-8578-6f625401d1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.156577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa231d1-ba7c-436b-8578-6f625401d1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.225803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8fa231d1-ba7c-436b-8578-6f625401d1b4 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:16.227099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8fa231d1-ba7c-436b-8578-6f625401d1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.331688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 728/1472] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:16.419534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40161e-f32d-4f53-8581-f7a342bd7e18 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.419820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40161e-f32d-4f53-8581-f7a342bd7e18 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.420093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40161e-f32d-4f53-8581-f7a342bd7e18 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.422097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 745/1473] 10.222.0.58 () {54 vars in 966 bytes} [Thu Jul 21 22:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:16.427529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3e31-911a-40b1-a0eb-360162c60787 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.427867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3e31-911a-40b1-a0eb-360162c60787 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.428184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3e31-911a-40b1-a0eb-360162c60787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.506722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b0ec3e31-911a-40b1-a0eb-360162c60787 None None] MFA Rules not processed for user `3df4115fcce342f2ae095ec1a2af4cca`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:16.507968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b0ec3e31-911a-40b1-a0eb-360162c60787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.529524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:16.586077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 729/1474] 10.222.0.58 () {58 vars in 1063 bytes} [Thu Jul 21 22:27:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:16.594842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40161e-f32d-4f53-8581-f7a342bd7e18 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:16.596427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6a40161e-f32d-4f53-8581-f7a342bd7e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.704807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40161e-f32d-4f53-8581-f7a342bd7e18 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:16.782386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:16.783147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.783444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.783724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.785536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.847345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:16.907760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:16.909215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b861e57-46cc-4cd0-9a9a-49fee5221e4e service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:16.928738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 746/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:19.147531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b9cb5-1060-403b-ae05-583448fbe430 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:19.148137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b9cb5-1060-403b-ae05-583448fbe430 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:19.148137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b9cb5-1060-403b-ae05-583448fbe430 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:19.149494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 730/1476] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:19.158278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f14ef-ba6a-4732-ae0b-7bf2265447fd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:19.158828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f14ef-ba6a-4732-ae0b-7bf2265447fd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:19.158828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f14ef-ba6a-4732-ae0b-7bf2265447fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:19.229742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-572f14ef-ba6a-4732-ae0b-7bf2265447fd None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:19.231447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-572f14ef-ba6a-4732-ae0b-7bf2265447fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:19.349994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 747/1477] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:19.373803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b9cb5-1060-403b-ae05-583448fbe430 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:19.376009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9b5b9cb5-1060-403b-ae05-583448fbe430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:19.526574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b9cb5-1060-403b-ae05-583448fbe430 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:19.583134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:19.632888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:19.633554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:19.633834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:19.634116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:19.634857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:19.636697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:27:19.636697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:27:19.636697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:27:19.637857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce1afd-809e-468d-90ab-a4c1f763b103 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:19.657724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 731/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jul 21 22:27:19 2022] GET /identity/v3/projects => generated 2553 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:27:21.757263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31586a-9661-4c7f-818b-4d2686c38849 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:21.757851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31586a-9661-4c7f-818b-4d2686c38849 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:21.757851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31586a-9661-4c7f-818b-4d2686c38849 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:21.759513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 748/1479] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:21.764976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e59c01-d60c-4a39-b5b5-30d9c5fb3081 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:21.765544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e59c01-d60c-4a39-b5b5-30d9c5fb3081 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:21.765892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e59c01-d60c-4a39-b5b5-30d9c5fb3081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:21.838646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-66e59c01-d60c-4a39-b5b5-30d9c5fb3081 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:21.839915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-66e59c01-d60c-4a39-b5b5-30d9c5fb3081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:21.944983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 732/1480] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:21.991457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31586a-9661-4c7f-818b-4d2686c38849 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:21.993021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8a31586a-9661-4c7f-818b-4d2686c38849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:22.003218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:22.154246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31586a-9661-4c7f-818b-4d2686c38849 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:22.243851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:22.279166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:22.279819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:22.280096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:22.281057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:22.282721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:22.372135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:22.373742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb753ea-7656-4992-bce7-b607cbd0d321 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:22.392807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 749/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:24.706967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbca7-7813-45b4-be00-874416059857 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:24.707655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbca7-7813-45b4-be00-874416059857 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:24.707655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbca7-7813-45b4-be00-874416059857 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:24.708952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 733/1482] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:24.714517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a028e-f77f-460c-855d-2fa724eda915 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:24.714981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a028e-f77f-460c-855d-2fa724eda915 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:24.715641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a028e-f77f-460c-855d-2fa724eda915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:24.793293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d67a028e-f77f-460c-855d-2fa724eda915 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:24.794540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d67a028e-f77f-460c-855d-2fa724eda915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:24.849427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:24.905740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 750/1483] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:24.942434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbca7-7813-45b4-be00-874416059857 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:24.944045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-751cbca7-7813-45b4-be00-874416059857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:24.969466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:25.053748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbca7-7813-45b4-be00-874416059857 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:25.131675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:25.132336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:25.132621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:25.132891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:25.134595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:25.215259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:25.251837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:25.253323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2ad1e-7901-44c5-96b4-b0a4ca5027a8 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:25.391426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 734/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:27.528927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0da0a3-aa3d-4950-9e04-bf4a23e0eb72 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:27.529614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0da0a3-aa3d-4950-9e04-bf4a23e0eb72 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:27.529614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0da0a3-aa3d-4950-9e04-bf4a23e0eb72 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:27.532027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 751/1485] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:27.538036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c4bc4-ebce-4a19-ac3e-29b8256f631c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:27.538590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c4bc4-ebce-4a19-ac3e-29b8256f631c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:27.538590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c4bc4-ebce-4a19-ac3e-29b8256f631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:27.608161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-072c4bc4-ebce-4a19-ac3e-29b8256f631c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:27.609523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-072c4bc4-ebce-4a19-ac3e-29b8256f631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:27.713797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 735/1486] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:27.753228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0da0a3-aa3d-4950-9e04-bf4a23e0eb72 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:27.754772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe0da0a3-aa3d-4950-9e04-bf4a23e0eb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:27.865403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0da0a3-aa3d-4950-9e04-bf4a23e0eb72 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:27.939358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:27.941297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:27.942023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:27.942334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:27.942615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:27.944917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:28.055158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:28.056625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0c384-a0d5-4af6-ac0f-6dcc6c39396b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:28.075906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 752/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:30.216731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5abb-1435-463a-a163-5b41ce857193 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:30.217371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5abb-1435-463a-a163-5b41ce857193 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:30.217371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b5abb-1435-463a-a163-5b41ce857193 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:30.218968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 736/1488] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:30.224439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e204e-e76d-4b8b-bceb-5a86f9eae08c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:30.224888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e204e-e76d-4b8b-bceb-5a86f9eae08c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:30.225038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e204e-e76d-4b8b-bceb-5a86f9eae08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:30.290560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cc9e204e-e76d-4b8b-bceb-5a86f9eae08c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:30.291793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cc9e204e-e76d-4b8b-bceb-5a86f9eae08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:30.344492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:30.395685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 753/1489] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:30.440093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5abb-1435-463a-a163-5b41ce857193 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:30.441756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-151b5abb-1435-463a-a163-5b41ce857193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:30.486070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:30.558361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b5abb-1435-463a-a163-5b41ce857193 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:30.638501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:30.639192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:30.639480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:30.639779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:30.641615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:30.740957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:30.775816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:30.777292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd79585-10de-4ebf-8976-f763c5c5bd7f service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:30.801349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 737/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:33.183223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950966e8-e69e-4982-97d1-a5e72d6979a7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:33.183806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950966e8-e69e-4982-97d1-a5e72d6979a7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:33.183806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950966e8-e69e-4982-97d1-a5e72d6979a7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:33.185148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 754/1491] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jul 21 22:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:33.190750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9361b2-3baf-4e82-8e53-3656878957c2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:33.191251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9361b2-3baf-4e82-8e53-3656878957c2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:33.191358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9361b2-3baf-4e82-8e53-3656878957c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:33.257038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6d9361b2-3baf-4e82-8e53-3656878957c2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:33.258253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6d9361b2-3baf-4e82-8e53-3656878957c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:33.361327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 738/1492] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:33.400611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950966e8-e69e-4982-97d1-a5e72d6979a7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:33.402177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-950966e8-e69e-4982-97d1-a5e72d6979a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:33.514962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950966e8-e69e-4982-97d1-a5e72d6979a7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:33.596248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:33.596904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:33.597187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:33.597464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:33.599161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:33.608416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:33.712218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:33.713678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb77cc-9907-408b-9f21-3e3b62d04043 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:33.734422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 755/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:43.359045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=82 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:43.731229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=50 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:01.853157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:01.853663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:01.853780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:01.929027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:01.930154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:02.032519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 739/1494] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:02.040765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:02.041289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:02.041289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:02.115315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:02.116512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:02.215142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 756/1495] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:02.220885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:02.222095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:02.236939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3bb61-6940-402b-b83b-ee53e5ee0044 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:02.299676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:02.334228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:02.334916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:02.335231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:02.335505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:02.336229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:29:02.336533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:02.337547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:02.337547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:29:02.337547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:29:02.338429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:02.350576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 740/1496] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:29:02.359547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:02.368817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c9e95-19dc-45b3-824f-952c5d9bab80 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:02.430524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:02.466353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:02.467512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:02.467983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:02.468265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:02.468960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:02.470725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:02.470725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:29:02.470725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:29:02.471972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:02.529179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 757/1497] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:29:02 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:29:02.533538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:02.535130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:02.570359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37845-9b97-48aa-b352-2bec8085872e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:02.664569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:02.665240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:02.665518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:02.665785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:02.666460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:02.667521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:02.667521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:29:02.667521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:29:02.668707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:02.710667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 741/1498] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:29:02 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:29:02.714858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:02.716958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:02.784380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971b4da-cf21-48e5-8023-6681479dae9d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:02.867160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:02.879005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:02.879765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:02.880046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:02.880318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:02.881038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:02.882049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:02.882049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:29:02.882049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:29:02.882952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:02.895297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 758/1499] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:29:02.899558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:02.901138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:02.913494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c815ac-5021-4f0f-b878-88a4b8868b50 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:02.939554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:03.011925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:03.012545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.012824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.013095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.014354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:03.015413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:03.015413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:29:03.015413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:29:03.016299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3014a9-27c9-4ccf-ab7c-427db399a33f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:03.028785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 742/1500] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:29:03.033672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:03.035263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:03.068854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a9baf-7905-4e4c-8b85-a3d3a57e39ad admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:03.163818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:03.164956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.165261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.165556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] PATH_INFO: `/v3/projects/9e56e8eb48574299bd8ed6996d3c9677/users/e77c35a8707a4e2abcc300b03ab11d44/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.190148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e56e8eb48574299bd8ed6996d3c9677, user_id=e77c35a8707a4e2abcc300b03ab11d44, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:03.191235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:03.191235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:29:03.191235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:29:03.193201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad790699-3932-4acc-b57d-6eb69a242e29 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:03.222639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 759/1501] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:29:03 2022] PUT /identity/v3/projects/9e56e8eb48574299bd8ed6996d3c9677/users/e77c35a8707a4e2abcc300b03ab11d44/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:29:03.235247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7aef4-4c44-4c0a-8947-40f8ffca4a6a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.235541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7aef4-4c44-4c0a-8947-40f8ffca4a6a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.235824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7aef4-4c44-4c0a-8947-40f8ffca4a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.292135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-61b7aef4-4c44-4c0a-8947-40f8ffca4a6a None None] MFA Rules not processed for user `e77c35a8707a4e2abcc300b03ab11d44`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:03.293279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-61b7aef4-4c44-4c0a-8947-40f8ffca4a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:03.371368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 743/1502] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:29:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:29:03.384601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e6752-06d6-4322-bf01-45fbe8129ee4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.384895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e6752-06d6-4322-bf01-45fbe8129ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.385178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e6752-06d6-4322-bf01-45fbe8129ee4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.386712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 760/1503] 10.222.0.58 () {54 vars in 964 bytes} [Thu Jul 21 22:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:29:03.392924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a21c8-51e4-4522-a52e-097e45df98bd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.393217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a21c8-51e4-4522-a52e-097e45df98bd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.393500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a21c8-51e4-4522-a52e-097e45df98bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.424686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:03.473057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ef3a21c8-51e4-4522-a52e-097e45df98bd None None] MFA Rules not processed for user `d529f6af558a44ef909937d5a960cb88`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:03.474284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ef3a21c8-51e4-4522-a52e-097e45df98bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:03.553659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 744/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Jul 21 22:29:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:29:03.561825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e6752-06d6-4322-bf01-45fbe8129ee4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:03.563390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7d4e6752-06d6-4322-bf01-45fbe8129ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:03.613730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:03.669437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e6752-06d6-4322-bf01-45fbe8129ee4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:03.746972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:03.747678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.748007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.748519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.749920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:03.858136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:03.859629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:03.863138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:03.878142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 761/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:29:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:29:03.921899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6176b2-819a-4e02-a6e2-e3bd7bf276f2 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:03.922460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6176b2-819a-4e02-a6e2-e3bd7bf276f2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:03.922460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6176b2-819a-4e02-a6e2-e3bd7bf276f2 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:03.923863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 745/1506] 10.222.0.58 () {52 vars in 829 bytes} [Thu Jul 21 22:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:29:03.927592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:03.928847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:04.031485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3eddb-b755-41da-a8c3-c892010f3174 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:04.126188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:04.127168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:04.127458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:04.127736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] PATH_INFO: `/v3/users/e77c35a8707a4e2abcc300b03ab11d44` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:04.140431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77c35a8707a4e2abcc300b03ab11d44)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:04.141459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:04.141459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:29:04.141459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:29:04.142676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:04.152976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:04.212773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77c35a8707a4e2abcc300b03ab11d44'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:04.223400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77c35a8707a4e2abcc300b03ab11d44'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:04.242043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77c35a8707a4e2abcc300b03ab11d44'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:04.242517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77c35a8707a4e2abcc300b03ab11d44'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:04.254769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77c35a8707a4e2abcc300b03ab11d44'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:04.260691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 762/1507] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:29:03 2022] DELETE /identity/v3/users/e77c35a8707a4e2abcc300b03ab11d44 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:29:04.265414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:04.265894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:04.266176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:04.325072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:04.326335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:04.400989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:04.414950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 746/1508] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:04.426554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:04.428731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:04.530723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1a5f-b640-475b-81d9-647c2d195371 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:04.585505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:04.608696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:04.609359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:04.609645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:04.609928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:04.611635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:04.722420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:04.723908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3b1d9-acc3-4d87-ac2b-ee12531d956a service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:04.742355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 763/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:29:05.078674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:05.079944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:05.091495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87452b23-d5d7-4593-8c57-425b6ee2b21e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:05.191195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:05.191977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:05.192259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:05.192532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] PATH_INFO: `/v3/projects/9e56e8eb48574299bd8ed6996d3c9677` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:05.203515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e56e8eb48574299bd8ed6996d3c9677)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:05.204552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:05.204552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:29:05.204552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:29:05.205759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:05.264207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e56e8eb48574299bd8ed6996d3c9677', 'request_id': 'req-08a93a1d-1fb6-49bc-a903-f72c786a4980'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:05.273978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e56e8eb48574299bd8ed6996d3c9677', 'request_id': 'req-08a93a1d-1fb6-49bc-a903-f72c786a4980'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:05.288780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e56e8eb48574299bd8ed6996d3c9677', 'request_id': 'req-08a93a1d-1fb6-49bc-a903-f72c786a4980'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:05.313438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] The token cache is being invalidate because project 9e56e8eb48574299bd8ed6996d3c9677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:29:05.313729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:05.314264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08a93a1d-1fb6-49bc-a903-f72c786a4980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:05.315652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 747/1510] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:29:05 2022] DELETE /identity/v3/projects/9e56e8eb48574299bd8ed6996d3c9677 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:29:14.739957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=62 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:15.311548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:30:43.554175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510443b-c17d-45e7-a5bd-fc1c139e0275 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:43.554729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510443b-c17d-45e7-a5bd-fc1c139e0275 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:43.554729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510443b-c17d-45e7-a5bd-fc1c139e0275 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:43.556188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:30:43.561127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249d749-a762-497a-95fa-40945979fcdc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:43.561718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249d749-a762-497a-95fa-40945979fcdc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:43.561825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249d749-a762-497a-95fa-40945979fcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:43.654079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0249d749-a762-497a-95fa-40945979fcdc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:30:43.655889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0249d749-a762-497a-95fa-40945979fcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:43.775490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:30:43.940194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510443b-c17d-45e7-a5bd-fc1c139e0275 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:30:43.942772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7510443b-c17d-45e7-a5bd-fc1c139e0275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:44.077900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510443b-c17d-45e7-a5bd-fc1c139e0275 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:30:44.083428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf3004-c9c7-489d-b6b9-790e315dc48e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:30:44.084077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf3004-c9c7-489d-b6b9-790e315dc48e service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:44.084355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf3004-c9c7-489d-b6b9-790e315dc48e service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:44.084633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf3004-c9c7-489d-b6b9-790e315dc48e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:44.093591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf3004-c9c7-489d-b6b9-790e315dc48e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:30:44.095213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf3004-c9c7-489d-b6b9-790e315dc48e service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:30:44.107003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 765/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:30:44.973914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002cb7-9198-4b15-a7cf-1ff12fb2ab2a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:44.974482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002cb7-9198-4b15-a7cf-1ff12fb2ab2a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:44.974482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002cb7-9198-4b15-a7cf-1ff12fb2ab2a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:44.975856 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:30:44.980831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308276ca-8437-4b6d-b4d9-48e6e09a5992 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:44.981298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308276ca-8437-4b6d-b4d9-48e6e09a5992 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:44.981400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308276ca-8437-4b6d-b4d9-48e6e09a5992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:45.000929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-308276ca-8437-4b6d-b4d9-48e6e09a5992 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:30:45.002121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-308276ca-8437-4b6d-b4d9-48e6e09a5992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:45.010611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:30:45.168171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16002cb7-9198-4b15-a7cf-1ff12fb2ab2a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:30:45.183358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16002cb7-9198-4b15-a7cf-1ff12fb2ab2a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:30:45.189611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d474dd-71d4-456d-b032-b9d449175b49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:30:45.190268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d474dd-71d4-456d-b032-b9d449175b49 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:45.190555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d474dd-71d4-456d-b032-b9d449175b49 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:45.190826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d474dd-71d4-456d-b032-b9d449175b49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:45.195086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8562a-2cc2-4c16-8280-61857e4f72ee None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:45.195373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8562a-2cc2-4c16-8280-61857e4f72ee None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:45.195656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8562a-2cc2-4c16-8280-61857e4f72ee None None] PATH_INFO: `/v3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:45.197245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 767/1516] 10.222.0.13 () {54 vars in 925 bytes} [Thu Jul 21 22:30:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:30:45.200282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d474dd-71d4-456d-b032-b9d449175b49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:30:45.202605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d474dd-71d4-456d-b032-b9d449175b49 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:30:45.204301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d3046-8f95-4297-b125-4bff34e2b215 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:45.204594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d3046-8f95-4297-b125-4bff34e2b215 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:45.204877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d3046-8f95-4297-b125-4bff34e2b215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:45.213496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 750/1517] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:30:45.277396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-487d3046-8f95-4297-b125-4bff34e2b215 None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:30:45.278652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-487d3046-8f95-4297-b125-4bff34e2b215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:45.354247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:30:45.376822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 768/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jul 21 22:30:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:30:49.221104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13798e9-af10-47c4-9bff-3ee872610664 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:49.221598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13798e9-af10-47c4-9bff-3ee872610664 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:49.221598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13798e9-af10-47c4-9bff-3ee872610664 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:49.222954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 751/1519] 10.222.0.58 () {54 vars in 961 bytes} [Thu Jul 21 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:30:49.228084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1f7e6-399c-4c13-8977-73112f2ec485 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:49.228541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1f7e6-399c-4c13-8977-73112f2ec485 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:49.228811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1f7e6-399c-4c13-8977-73112f2ec485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:49.247729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c3e1f7e6-399c-4c13-8977-73112f2ec485 None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:30:49.249590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c3e1f7e6-399c-4c13-8977-73112f2ec485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:49.258564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 769/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Jul 21 22:30:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:30:49.266968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13798e9-af10-47c4-9bff-3ee872610664 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:30:49.282723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13798e9-af10-47c4-9bff-3ee872610664 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:30:49.289157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf6c42-28eb-4e25-b1a7-4694f24fda0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:30:49.289899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf6c42-28eb-4e25-b1a7-4694f24fda0b service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:49.290180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf6c42-28eb-4e25-b1a7-4694f24fda0b service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:49.290471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf6c42-28eb-4e25-b1a7-4694f24fda0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:49.300901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf6c42-28eb-4e25-b1a7-4694f24fda0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:30:49.302396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf6c42-28eb-4e25-b1a7-4694f24fda0b service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:30:49.313563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 752/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:30:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:30:56.595216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b69d2f-4f61-46c5-9cde-174a6b5d2a8e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:56.595799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b69d2f-4f61-46c5-9cde-174a6b5d2a8e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:56.595799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b69d2f-4f61-46c5-9cde-174a6b5d2a8e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:56.597127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:30:56.602075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4b77b-ba7d-4faf-9d76-c9dd369f488a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:56.602493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4b77b-ba7d-4faf-9d76-c9dd369f488a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:56.602647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4b77b-ba7d-4faf-9d76-c9dd369f488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:56.620406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b5d4b77b-ba7d-4faf-9d76-c9dd369f488a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:30:56.621614 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b5d4b77b-ba7d-4faf-9d76-c9dd369f488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:56.630148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:30:56.785840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b69d2f-4f61-46c5-9cde-174a6b5d2a8e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:30:56.801054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b69d2f-4f61-46c5-9cde-174a6b5d2a8e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:30:56.806536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5e087-497e-43ba-a2d3-c234f93542a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:30:56.807220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e087-497e-43ba-a2d3-c234f93542a7 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:56.807510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e087-497e-43ba-a2d3-c234f93542a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:56.807789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e087-497e-43ba-a2d3-c234f93542a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:56.816845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5e087-497e-43ba-a2d3-c234f93542a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:30:56.818443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5e087-497e-43ba-a2d3-c234f93542a7 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:30:56.828329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 771/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:30:57.662478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c453420-58a1-4187-885d-630cddc3d548 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:57.662478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c453420-58a1-4187-885d-630cddc3d548 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:57.662958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c453420-58a1-4187-885d-630cddc3d548 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:57.664036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:30:57.668973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693ef75-9aa1-4fcb-9c33-a031b73ca529 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:57.669379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693ef75-9aa1-4fcb-9c33-a031b73ca529 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:57.669538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693ef75-9aa1-4fcb-9c33-a031b73ca529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:57.687286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3693ef75-9aa1-4fcb-9c33-a031b73ca529 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:30:57.688499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3693ef75-9aa1-4fcb-9c33-a031b73ca529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:30:57.696383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:30:57.852593 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c453420-58a1-4187-885d-630cddc3d548 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:30:57.866849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c453420-58a1-4187-885d-630cddc3d548 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:30:57.872362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951c634-ee41-47c0-9ce6-4439c1ed65ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:30:57.872993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c634-ee41-47c0-9ce6-4439c1ed65ed service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:30:57.873369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c634-ee41-47c0-9ce6-4439c1ed65ed service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:30:57.873914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c634-ee41-47c0-9ce6-4439c1ed65ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:30:57.881734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951c634-ee41-47c0-9ce6-4439c1ed65ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:30:57.883419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951c634-ee41-47c0-9ce6-4439c1ed65ed service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:30:57.893543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 755/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:02.074920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a02a-cda6-4d9c-a406-09cad62b2130 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:02.075539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a02a-cda6-4d9c-a406-09cad62b2130 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:02.075539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a02a-cda6-4d9c-a406-09cad62b2130 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:02.077403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:02.082651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cc1f4-5491-4f08-af5c-b5a29080ad10 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:02.083162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cc1f4-5491-4f08-af5c-b5a29080ad10 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:02.083283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cc1f4-5491-4f08-af5c-b5a29080ad10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:02.102059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f11cc1f4-5491-4f08-af5c-b5a29080ad10 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:02.103290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f11cc1f4-5491-4f08-af5c-b5a29080ad10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:02.111390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:02.184735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a02a-cda6-4d9c-a406-09cad62b2130 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:02.186961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cea6a02a-cda6-4d9c-a406-09cad62b2130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:02.268969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a02a-cda6-4d9c-a406-09cad62b2130 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:02.272717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0d4db-97ea-4312-ae93-41d7669ada36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:02.273366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0d4db-97ea-4312-ae93-41d7669ada36 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:02.273649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0d4db-97ea-4312-ae93-41d7669ada36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:02.273926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0d4db-97ea-4312-ae93-41d7669ada36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:02.283918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0d4db-97ea-4312-ae93-41d7669ada36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:02.285366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0d4db-97ea-4312-ae93-41d7669ada36 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:02.296427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 774/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:04.353231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5bbe8-b4d8-4b5b-ad55-eb5a80affea3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:04.353821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5bbe8-b4d8-4b5b-ad55-eb5a80affea3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:04.353821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5bbe8-b4d8-4b5b-ad55-eb5a80affea3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:04.355229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:04.361441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b666780-77dc-4673-aa7a-688565c2f81f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:04.361911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b666780-77dc-4673-aa7a-688565c2f81f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:04.362020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b666780-77dc-4673-aa7a-688565c2f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:04.381269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4b666780-77dc-4673-aa7a-688565c2f81f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:04.382480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4b666780-77dc-4673-aa7a-688565c2f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:04.391445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:04.428145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5bbe8-b4d8-4b5b-ad55-eb5a80affea3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:04.440202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5bbe8-b4d8-4b5b-ad55-eb5a80affea3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:04.444062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca1a1a-a065-409d-a64d-1a4a0d0dedf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:04.444719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1a1a-a065-409d-a64d-1a4a0d0dedf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:04.445022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1a1a-a065-409d-a64d-1a4a0d0dedf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:04.445296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1a1a-a065-409d-a64d-1a4a0d0dedf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:04.455431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca1a1a-a065-409d-a64d-1a4a0d0dedf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:04.456881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca1a1a-a065-409d-a64d-1a4a0d0dedf4 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:04.468204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 758/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:31:08.233743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012473b-e5e6-4758-8413-c0f3f42fdded None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:08.234311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012473b-e5e6-4758-8413-c0f3f42fdded None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:08.234311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012473b-e5e6-4758-8413-c0f3f42fdded None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:08.235725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:08.240881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886851e-dde0-4cbe-a996-da3607bd88a5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:08.241337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886851e-dde0-4cbe-a996-da3607bd88a5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:08.241443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886851e-dde0-4cbe-a996-da3607bd88a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:08.247810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:31:08.260249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2886851e-dde0-4cbe-a996-da3607bd88a5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:08.261469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2886851e-dde0-4cbe-a996-da3607bd88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:08.269980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:08.579621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012473b-e5e6-4758-8413-c0f3f42fdded None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:08.581326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8012473b-e5e6-4758-8413-c0f3f42fdded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:08.660767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012473b-e5e6-4758-8413-c0f3f42fdded None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:08.665040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6cd27-c36f-4e6f-9c71-8e91d1e5ea11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:08.665692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6cd27-c36f-4e6f-9c71-8e91d1e5ea11 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:08.665976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6cd27-c36f-4e6f-9c71-8e91d1e5ea11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:08.666253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6cd27-c36f-4e6f-9c71-8e91d1e5ea11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:08.675941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6cd27-c36f-4e6f-9c71-8e91d1e5ea11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:08.677378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6cd27-c36f-4e6f-9c71-8e91d1e5ea11 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:08.687427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 777/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:10.779965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f86439-8b77-4c3a-98e4-64c796bb376d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:10.780563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f86439-8b77-4c3a-98e4-64c796bb376d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:10.780563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f86439-8b77-4c3a-98e4-64c796bb376d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:10.781908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:10.787159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da989fe-7b66-475f-9a91-28508d3d5293 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:10.787581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da989fe-7b66-475f-9a91-28508d3d5293 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:10.787720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da989fe-7b66-475f-9a91-28508d3d5293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:10.805520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8da989fe-7b66-475f-9a91-28508d3d5293 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:10.806704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8da989fe-7b66-475f-9a91-28508d3d5293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:10.815333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:11.125562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f86439-8b77-4c3a-98e4-64c796bb376d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:11.127283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-09f86439-8b77-4c3a-98e4-64c796bb376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:11.140430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f86439-8b77-4c3a-98e4-64c796bb376d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:11.144045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aedc06-3cd1-4bf3-b429-1830bbd3d3c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:11.144685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aedc06-3cd1-4bf3-b429-1830bbd3d3c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:11.144970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aedc06-3cd1-4bf3-b429-1830bbd3d3c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:11.145251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aedc06-3cd1-4bf3-b429-1830bbd3d3c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:11.154710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aedc06-3cd1-4bf3-b429-1830bbd3d3c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:11.156238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aedc06-3cd1-4bf3-b429-1830bbd3d3c4 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:11.165534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 761/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:20.800528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:31:21.161415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:42.015989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.017053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.017541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.051692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.052953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.068112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 779/1540] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:32:42.077981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b7282-c6b7-4795-badb-f630e02fcc73 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.078566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b7282-c6b7-4795-badb-f630e02fcc73 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.078566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b7282-c6b7-4795-badb-f630e02fcc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.106821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-389b7282-c6b7-4795-badb-f630e02fcc73 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.108232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-389b7282-c6b7-4795-badb-f630e02fcc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.127580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 762/1541] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:32:42.132348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.148047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20bde-9167-40ab-b84b-5490c8e7858d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.157426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.158380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.158784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.159276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.159865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:42.160177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.161245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:32:42.161245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:32:42.161245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:32:42.162173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.165867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dba32a-8d4f-4db1-b0b4-c5d288cb2fee None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.166200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dba32a-8d4f-4db1-b0b4-c5d288cb2fee None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.166598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dba32a-8d4f-4db1-b0b4-c5d288cb2fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.182418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 780/1542] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:42.192260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.202292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d2dba32a-8d4f-4db1-b0b4-c5d288cb2fee None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.203637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d2dba32a-8d4f-4db1-b0b4-c5d288cb2fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.208707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12244e3-fca2-40c6-9045-674f714d78a8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.213657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.214586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.214879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.215457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.216214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.218171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.234293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 763/1543] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:32:42.242960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb799b-0e51-4c32-aa0f-72e819feff71 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.243304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb799b-0e51-4c32-aa0f-72e819feff71 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.243548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb799b-0e51-4c32-aa0f-72e819feff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.275444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 781/1544] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:42.276805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.281392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-47eb799b-0e51-4c32-aa0f-72e819feff71 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.282757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47eb799b-0e51-4c32-aa0f-72e819feff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.384123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf9f4a-0b2d-4b88-bf52-49b26ba24b66 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.389858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.390676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.390969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.391276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.391980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.393108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:32:42.393108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:32:42.393108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:32:42.394882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b36f0-9fc9-4403-80b6-99ec0a56240c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.400510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 764/1545] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.401843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.402136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.402417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.430935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 782/1546] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:42.436633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.436947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.437274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.442883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.444217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.468010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.468586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 765/1547] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.469226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.469434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.485212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3308c39-916d-4775-87c1-2809cca19333 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.486917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 783/1548] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.488130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.494157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.494901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.495224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.495504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.496301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:42.496610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.498349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464365-c89e-4cfa-89fa-bca21b59f94a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.499509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41f66f-c49c-4fba-ac83-03af4268d228 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.518038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6ab05-25e4-484c-a8ad-ac4e89a68e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.518038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ab05-25e4-484c-a8ad-ac4e89a68e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.518038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ab05-25e4-484c-a8ad-ac4e89a68e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.518038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ab05-25e4-484c-a8ad-ac4e89a68e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.518950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6ab05-25e4-484c-a8ad-ac4e89a68e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.520511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6ab05-25e4-484c-a8ad-ac4e89a68e11 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.538663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 766/1549] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:42.540851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9ffd3-6188-447c-b425-6b1fb91b4e8f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.543009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9ffd3-6188-447c-b425-6b1fb91b4e8f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.547686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9ffd3-6188-447c-b425-6b1fb91b4e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.552474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 784/1550] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:42.554083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.554598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.555182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.584741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ccd9ffd3-6188-447c-b425-6b1fb91b4e8f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.585975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ccd9ffd3-6188-447c-b425-6b1fb91b4e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.587182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.588379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.597492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 767/1551] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.598575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 785/1552] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.599004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.599408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.599516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.599729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.612996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43676f-e3d0-4b00-be98-8c4b17938717 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.618266 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f2934-f614-4d0c-aaa9-4c9c583b2318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.618937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f2934-f614-4d0c-aaa9-4c9c583b2318 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.619256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f2934-f614-4d0c-aaa9-4c9c583b2318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.619540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f2934-f614-4d0c-aaa9-4c9c583b2318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.620240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f2934-f614-4d0c-aaa9-4c9c583b2318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.621393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.622843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f2934-f614-4d0c-aaa9-4c9c583b2318 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.623896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.631939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 768/1553] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.632765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.646482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf451fb-ece5-435d-b0da-1bfd1e4662cc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.651936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.652638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.652919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.653193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.654235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.655699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 786/1554] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:42.659185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.659324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.659324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.659324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.670631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 769/1555] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:42.670631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.678751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975807d5-e129-470b-b1c5-0e12fa965e37 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.681600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.682819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.737614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:42.738293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.739046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.739338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.739613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.740351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:42.740688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.742453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1559b-c93f-4d5e-852d-c66b703fc00a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.752112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 787/1556] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.752878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.756425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 770/1557] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:42.760167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b6a49-67a0-4dbf-9d88-9fdcef4f88c5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.760615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b6a49-67a0-4dbf-9d88-9fdcef4f88c5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.761161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b6a49-67a0-4dbf-9d88-9fdcef4f88c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.764665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b97f72-8119-4ec7-b668-c53d68b704ca None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.769192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.769906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.770182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.770461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.771233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:42.771543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.773425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051526bb-15e3-42a8-abd6-979d148f52cb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.782031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bb5b6a49-67a0-4dbf-9d88-9fdcef4f88c5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.783338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bb5b6a49-67a0-4dbf-9d88-9fdcef4f88c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.786188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 788/1558] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:42.788412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.788706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.788992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.791671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 771/1559] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.792927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.793214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.793555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.807848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.809062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.812575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.813774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.820405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 789/1560] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.820405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.829058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 772/1561] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.830060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.837967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61b94b-93eb-40e1-914f-9462a5ea344f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.840658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ea20-46b1-4254-af8c-323ff47bd036 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.845401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bf4c3-9188-4b31-accb-077a64e48834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.846083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bf4c3-9188-4b31-accb-077a64e48834 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.846363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bf4c3-9188-4b31-accb-077a64e48834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.846512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.846701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bf4c3-9188-4b31-accb-077a64e48834 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.847423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bf4c3-9188-4b31-accb-077a64e48834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.847536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.847708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.847987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] PATH_INFO: `/v3/projects/376889a38a46462d9ba21b0f94e45a1a/users/4915c900a2bc460283d237e6c8005117/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.849938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bf4c3-9188-4b31-accb-077a64e48834 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.865072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376889a38a46462d9ba21b0f94e45a1a, user_id=4915c900a2bc460283d237e6c8005117, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.866460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:32:42.866460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:32:42.866460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:32:42.868499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.874690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 790/1562] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/users => generated 294 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:42.876267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.876562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.876847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.882297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:42.889364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 773/1563] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:42 2022] PUT /identity/v3/projects/376889a38a46462d9ba21b0f94e45a1a/users/4915c900a2bc460283d237e6c8005117/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:42.890275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.898230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:42.899469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:42.961477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 791/1564] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:42.962224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:42.969367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bb510-20a4-48df-be49-db8fb3aa9437 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.973828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbe159-25ed-418a-b62f-3c671f39740e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:42.975331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.976015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.976333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.976589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.977328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.978410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:32:42.978410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:32:42.978410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:32:42.978693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c32d0-e0ab-4714-9fb2-9547735e36cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:42.979099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c32d0-e0ab-4714-9fb2-9547735e36cc admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:42.979369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c32d0-e0ab-4714-9fb2-9547735e36cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:42.979750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c32d0-e0ab-4714-9fb2-9547735e36cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:42.979868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cd260-2801-4392-b5d3-cdc561373f36 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:42.980329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c32d0-e0ab-4714-9fb2-9547735e36cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:42.982499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c32d0-e0ab-4714-9fb2-9547735e36cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.007583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 774/1565] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.008652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 792/1566] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:42 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.009207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.009574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.009854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.010902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.011383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.011819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.031818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:43.033026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:43.033500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:43.034753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:43.097702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 793/1567] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:43.098644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.100186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 775/1568] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:43.102260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.110044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9813b-4152-4e22-9f5c-00d0ebc18071 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.112031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9c43b-02f9-4eb2-bf7f-033ed62d1376 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.115265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.116021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.116358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.116728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.116904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1955-d7ae-48f4-a433-8fa60a9dbe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.117427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1955-d7ae-48f4-a433-8fa60a9dbe2f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.117553 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:43.117713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1955-d7ae-48f4-a433-8fa60a9dbe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.117992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1955-d7ae-48f4-a433-8fa60a9dbe2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.118092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.118730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c1955-d7ae-48f4-a433-8fa60a9dbe2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.119891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.120366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c1955-d7ae-48f4-a433-8fa60a9dbe2f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.131833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 776/1569] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.132968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.133263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.133547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.147239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 794/1570] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:43.147239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.149860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc11dd-6c78-48b3-9e8c-ea60ec3a587b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.155872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.156548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.156834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.157108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.157840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:43.158150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.159804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.173675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] MFA Rules not processed for user `4915c900a2bc460283d237e6c8005117`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:43.179267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:43.179401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 795/1571] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:43.180188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.192331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcfda-0f82-4c0d-952c-48cfeaa35930 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.197073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.197749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.198036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.198315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.199009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.201536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.209222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:43.224709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 796/1572] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.225527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.238571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e7d0a-5bb6-4210-ab11-ef34bddcf1a7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.243211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.243888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.244170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.244520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.245140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.247285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.250285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 777/1573] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:43.251171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.259561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:43.270003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0da553-ded8-42f7-ba4b-ea68901454aa None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.272435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 797/1574] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/users => generated 286 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.273264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.275584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.276363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.276653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.276919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.277705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:43.278021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.279814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.283797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d90da9-3de3-43ae-80d2-f910963a060d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.290396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.291075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.291379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.291655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.292387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:43.292682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.293318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 778/1575] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:43.294713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.294905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.306961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603013a-1fef-4006-a013-d72465dd5ceb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.312508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.313173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.313455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.313728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.314454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.314588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 798/1576] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:43.315457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.316140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.327086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67431afc-b7f1-461d-8d18-987c30a9e933 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.327861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 779/1577] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.328638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.334798 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.335504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.335784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.336059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.336745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.338426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664041d-c763-4f90-aca8-1bc6ee34066d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.338675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.344062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.344876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.345156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.345435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.346123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.348065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.365499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 799/1578] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.366381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.380106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 780/1579] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.381028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.436400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3424-2382-491e-8f32-64653f409233 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.443989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d9528-6d14-47a1-853b-63df5ecb4d9c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.447974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.448842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.449335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.449616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.492868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.493567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.493850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.494122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.494841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.496479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.506679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 800/1580] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.508308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.510381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.511818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.519083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f04b2-8c82-466b-b6a9-1b11f6ca002c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.521526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 781/1581] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:43.522573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.524825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.525686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.526102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.526386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.527140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.529588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.533609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d6f6-5c8c-4c90-8e78-cfec9bde9d2f service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.538181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.538839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.539624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.539900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.540596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.542486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 801/1582] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.542654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.543408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.554069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905e8ba-d5dc-4163-a4e4-311a5b7745a6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.558679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.559461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.560365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.560793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.561472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.563602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.573620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 782/1583] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.573620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.595321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 802/1584] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.596075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.607783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:43.637631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:43.655305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747f3f0-b060-4370-9eba-1ff2b2460ce5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.676407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db8cf-685d-4dd5-837d-725770c87c94 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.682202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.682893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.683217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.683731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.684209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.687086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.714957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 803/1585] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.716049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.726870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14698b84-0bfe-46a2-8a81-630170c11793 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.732260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.733083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.733365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.733636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.734334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.736300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.737342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.738782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.739279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.739726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] PATH_INFO: `/v3/projects/3b9849b45b2d414797c0cb368b787ddf/users/a6146c8083d04171863a4b603fd6a46c/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.760256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 804/1586] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.760566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9849b45b2d414797c0cb368b787ddf, user_id=a6146c8083d04171863a4b603fd6a46c, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.761268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.764565 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.776091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b80e6-f6f6-4b6c-bd49-ba6dbb27569f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.780965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 783/1587] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:43 2022] PUT /identity/v3/projects/3b9849b45b2d414797c0cb368b787ddf/users/a6146c8083d04171863a4b603fd6a46c/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:43.781808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.850193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.850936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.851268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.851601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.852348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.854018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.864819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85d45-f9cb-4338-881e-abd6327b4fe8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.865415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 805/1588] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.866295 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.874034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.875023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.875405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.875754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.876562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.878052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178b4c-4e12-4db3-9c74-096ac78aed5f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.878339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.882777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.883485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.883773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.884052 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.884743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.886843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.891121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 784/1589] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.892116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.904375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db824d24-f8b5-413c-bd08-035dd7d11e35 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.909304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.910185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.910541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.910885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.911668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.912717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 806/1590] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.913419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.913808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.926269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e40d7-7d75-4c34-9eb0-2796d431f408 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.932580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebafd2-b7d9-433c-8a5a-0637222dbe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.933263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebafd2-b7d9-433c-8a5a-0637222dbe26 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.933568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebafd2-b7d9-433c-8a5a-0637222dbe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.933835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebafd2-b7d9-433c-8a5a-0637222dbe26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.935147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebafd2-b7d9-433c-8a5a-0637222dbe26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.936969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebafd2-b7d9-433c-8a5a-0637222dbe26 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.941237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 785/1591] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/users => generated 279 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:43.941237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.949092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658857e-456f-49c8-89d5-32934da5dafc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.953793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 807/1592] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.955647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.955959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.956314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.956436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.956673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.956918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.957299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.958097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:43.959995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:43.972118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 786/1593] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:43.972911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:43.979535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:43.980757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:43.984911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c412525-4456-4dd7-ba55-ece0f022ed46 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:43.990319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:43.991244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:43.991414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 808/1594] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:43.994885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:43.994885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] PATH_INFO: `/v3/projects/3138cf3bb8a24ba19295565fca0a471e/users/cf3fd4428e3b46cd9e83e1d32571097b/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:43.995684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.008008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898adc-c923-48c1-b68f-360fc2661cda None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.015712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.016681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.017060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.017365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] PATH_INFO: `/v3/projects/75b642b9055c4317b3efbbfd1b3786dd/users/ea60343fe4be4f7983b0b6591bf79015/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.020624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3138cf3bb8a24ba19295565fca0a471e, user_id=cf3fd4428e3b46cd9e83e1d32571097b, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.023806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.036222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b642b9055c4317b3efbbfd1b3786dd, user_id=ea60343fe4be4f7983b0b6591bf79015, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.039285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.039416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 787/1595] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:43 2022] PUT /identity/v3/projects/3138cf3bb8a24ba19295565fca0a471e/users/cf3fd4428e3b46cd9e83e1d32571097b/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:44.043938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.072951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 809/1596] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:43 2022] PUT /identity/v3/projects/75b642b9055c4317b3efbbfd1b3786dd/users/ea60343fe4be4f7983b0b6591bf79015/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:44.072951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.099795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:44.105811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:44.158661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1bed6-42f2-4bf9-b043-578f724279fb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.163330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a1164-20b5-4700-9562-c61fea923316 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.164107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.164896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.165183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.165826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.166772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.170097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.170643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.171549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.172132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.172595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.173511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.175897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.182802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 788/1597] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:44.187458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.196460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 810/1598] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:44.197525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.201191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b23cf7-1ad9-442a-aede-f8e58d6c7c43 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.207032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.207754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.208038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.208311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.208733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4194d07-2733-4620-b38c-78c2ecfea326 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.209042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.210650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.214171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7b4d9-9345-46f2-84bf-0a80e6146974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.214849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7b4d9-9345-46f2-84bf-0a80e6146974 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.215176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7b4d9-9345-46f2-84bf-0a80e6146974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.215464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7b4d9-9345-46f2-84bf-0a80e6146974 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.216797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7b4d9-9345-46f2-84bf-0a80e6146974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.218465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7b4d9-9345-46f2-84bf-0a80e6146974 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.226710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 789/1599] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:44.227519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.235568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 811/1600] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:44.237965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33c92-ed76-447c-bd6b-dd07fddae317 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.238299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33c92-ed76-447c-bd6b-dd07fddae317 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.238655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d33c92-ed76-447c-bd6b-dd07fddae317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.258433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f8d33c92-ed76-447c-bd6b-dd07fddae317 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:44.259693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f8d33c92-ed76-447c-bd6b-dd07fddae317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:44.269646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 812/1601] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:44.270901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.271229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.271517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.304467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] MFA Rules not processed for user `ea60343fe4be4f7983b0b6591bf79015`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:44.305785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:44.310190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05836-1edc-45e4-aac5-129fd1fd56cc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.314374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.315030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.315364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.315644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.325181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.327196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.337108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 790/1602] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:44.338005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.350729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808890f7-f82a-4eaf-a746-a379836c1211 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.356313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.357394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.357686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.357967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.358704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.360505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.371506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 791/1603] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:44.372985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.384107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93387911-90cf-410b-a8a5-d3ce1ecef0db admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.387106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 813/1604] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:44.387106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.393397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.394432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.395089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.395590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.396476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.398576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.400441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097dc68-501c-4f53-a983-74b3744f4a5e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.407326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.408404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.408863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.409023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] PATH_INFO: `/v3/projects/d54636d00282483381454b9be56a43b5/users/1ee22783b3234c4f905fedce7d59f01d/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.415522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 792/1605] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:44.415522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.424377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f52b99-eaa2-4037-9a04-1f116eddf6eb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.430600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.431732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.432015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.432318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] PATH_INFO: `/v3/projects/11c72a831a934111b1c89a87f9bc81de/users/2a4393720e844b998ad3d309c06b22f9/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.440134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54636d00282483381454b9be56a43b5, user_id=1ee22783b3234c4f905fedce7d59f01d, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.444301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.456722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c72a831a934111b1c89a87f9bc81de, user_id=2a4393720e844b998ad3d309c06b22f9, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.460077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.462266 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 814/1606] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:44 2022] PUT /identity/v3/projects/d54636d00282483381454b9be56a43b5/users/1ee22783b3234c4f905fedce7d59f01d/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:44.465060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.477398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 793/1607] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:44 2022] PUT /identity/v3/projects/11c72a831a934111b1c89a87f9bc81de/users/2a4393720e844b998ad3d309c06b22f9/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:44.478552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.540099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f0209-254e-49af-b507-cd7bc6bedc70 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.555512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:44.571719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e713d-3be4-45de-9e8c-9285d9d870ac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.577345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.578395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.578682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.578956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] PATH_INFO: `/v3/projects/7f7d739ba4574964821462bf4be0e1f5/users/2c07a46f581b4bbb97ad99005471ebe3/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.592976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:44.599170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7d739ba4574964821462bf4be0e1f5, user_id=2c07a46f581b4bbb97ad99005471ebe3, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.601902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.605522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.606300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.606843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.607265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.618617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 794/1608] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:44 2022] PUT /identity/v3/projects/7f7d739ba4574964821462bf4be0e1f5/users/2c07a46f581b4bbb97ad99005471ebe3/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:44.618866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.619958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.620790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.633278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 815/1609] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:44.634464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.647533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5609eb-7bf9-4bdd-8e8f-45b801acc430 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.706632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7c278-844b-4d8c-9f42-9d219babf14c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.712353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3c435-a666-4802-aefa-31cfb3449b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.713056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3c435-a666-4802-aefa-31cfb3449b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.713435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3c435-a666-4802-aefa-31cfb3449b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.713626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3c435-a666-4802-aefa-31cfb3449b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.714400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3c435-a666-4802-aefa-31cfb3449b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.716565 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3c435-a666-4802-aefa-31cfb3449b51 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.720445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06049-65b2-41cf-8054-c32276b54ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.721769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06049-65b2-41cf-8054-c32276b54ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.722063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06049-65b2-41cf-8054-c32276b54ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.722339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06049-65b2-41cf-8054-c32276b54ce6 admin admin] PATH_INFO: `/v3/projects/58e5cbfc341048079874b818fcc619eb/users/3b054be28a9142618371ad0fc1d63510/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.739614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06049-65b2-41cf-8054-c32276b54ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e5cbfc341048079874b818fcc619eb, user_id=3b054be28a9142618371ad0fc1d63510, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.742959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06049-65b2-41cf-8054-c32276b54ce6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.757177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 795/1610] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:44.760025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e8392-8155-4c91-a57f-0b16a28949ee None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.760940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e8392-8155-4c91-a57f-0b16a28949ee None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.761134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e8392-8155-4c91-a57f-0b16a28949ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.771104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 816/1611] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:44 2022] PUT /identity/v3/projects/58e5cbfc341048079874b818fcc619eb/users/3b054be28a9142618371ad0fc1d63510/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:44.771104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.771104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.771104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.800076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] MFA Rules not processed for user `2a4393720e844b998ad3d309c06b22f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:44.801845 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8a2e8392-8155-4c91-a57f-0b16a28949ee None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:44.802503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:44.803530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a2e8392-8155-4c91-a57f-0b16a28949ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:44.888672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 796/1612] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:44.889616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac106b2-f31f-4225-8fa8-b02625a53f30 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.889917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac106b2-f31f-4225-8fa8-b02625a53f30 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.890327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac106b2-f31f-4225-8fa8-b02625a53f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.903149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 817/1613] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:44.904512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:44.921854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb9fe0-ac2d-4d53-9586-9c06171c7999 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:44.930627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab104b9-d773-4d2c-8204-dd48ff4dc3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:44.931939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4ac106b2-f31f-4225-8fa8-b02625a53f30 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:44.932936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab104b9-d773-4d2c-8204-dd48ff4dc3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.932936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab104b9-d773-4d2c-8204-dd48ff4dc3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.932936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab104b9-d773-4d2c-8204-dd48ff4dc3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.934073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab104b9-d773-4d2c-8204-dd48ff4dc3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:44.936473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4ac106b2-f31f-4225-8fa8-b02625a53f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:44.939255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab104b9-d773-4d2c-8204-dd48ff4dc3e6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:44.960595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 797/1614] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:44.969151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2d504-9e4a-4806-937d-165d2a3d9295 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.969531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2d504-9e4a-4806-937d-165d2a3d9295 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.969825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2d504-9e4a-4806-937d-165d2a3d9295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.976409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 818/1615] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/users => generated 292 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:44.979594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017966e-1bc2-49c3-a104-a41d05d7917d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:44.980431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017966e-1bc2-49c3-a104-a41d05d7917d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:44.980991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017966e-1bc2-49c3-a104-a41d05d7917d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:44.984261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 819/1616] 10.222.0.58 () {54 vars in 962 bytes} [Thu Jul 21 22:32:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:32:44.985697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017966e-1bc2-49c3-a104-a41d05d7917d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.016504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:45.045207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-54d2d504-9e4a-4806-937d-165d2a3d9295 None None] MFA Rules not processed for user `3b054be28a9142618371ad0fc1d63510`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:45.046445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-54d2d504-9e4a-4806-937d-165d2a3d9295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:45.100062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017966e-1bc2-49c3-a104-a41d05d7917d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.104378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.105202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.105666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.106928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.120296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.122690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.130541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:45.135723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 820/1617] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:45.136818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.140860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 798/1618] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:44 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:45.142300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.143126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.143827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.152615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8883ebd-f08e-40ca-9fb8-a7400bbe4564 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.162608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.163357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.163648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.164079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.164911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.167423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.182764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 821/1619] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:45.183590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.202879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab8da8-3ec1-4a3c-abc0-d85b6e1a4979 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.208994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.209675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.209961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.210239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.223904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.226178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.241374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 822/1620] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:45.242689 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.255380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:45.255857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6cc9-0cfd-44ba-bf36-7607156f3aa7 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.256637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:45.261172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.262191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.262482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.263827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.264597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.266680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.336282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 823/1621] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:45 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:45.340723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.437477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 799/1622] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Jul 21 22:32:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:45.446553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.458885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfc8c5-e29a-466f-8fbf-c743356b65a9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.505996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383d8ab-4d9f-4473-9bac-bee28caebd6c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.511698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.512618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.514515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.514983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] PATH_INFO: `/v3/projects/d28a2924960b484b8a80e4d6631c1119/users/ee1dec4ce04d43868d1c4318ec71ab62/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.534353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.535072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.535369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.535652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.536411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.538701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.565135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28a2924960b484b8a80e4d6631c1119, user_id=ee1dec4ce04d43868d1c4318ec71ab62, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.569263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.570348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 800/1623] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:45 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:45.575700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.599998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 824/1624] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:45 2022] PUT /identity/v3/projects/d28a2924960b484b8a80e4d6631c1119/users/ee1dec4ce04d43868d1c4318ec71ab62/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:45.601004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.769705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:45.770431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:45.779421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429d694-5912-4a0c-8901-3629441e843e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.782091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076a4c4-7f29-4534-a1a9-0273b373801c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.787562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.788274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.788567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.788851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.789695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.791446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.807864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 825/1625] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:45.812153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.829459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e224-f4d4-4dd6-bc7f-afea2a70664e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.839045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c685b5-4462-460d-a8cd-b9dfe2ebb8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.839877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c685b5-4462-460d-a8cd-b9dfe2ebb8dd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.840171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c685b5-4462-460d-a8cd-b9dfe2ebb8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.840584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c685b5-4462-460d-a8cd-b9dfe2ebb8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.841973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c685b5-4462-460d-a8cd-b9dfe2ebb8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.845312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c685b5-4462-460d-a8cd-b9dfe2ebb8dd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:45.878760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 826/1626] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:45.882601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29df2-2b52-45f2-abe8-4af2b3eefe82 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.884699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29df2-2b52-45f2-abe8-4af2b3eefe82 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.885637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29df2-2b52-45f2-abe8-4af2b3eefe82 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.898469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 827/1627] 10.222.0.58 () {54 vars in 962 bytes} [Thu Jul 21 22:32:45 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:32:45.899793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29df2-2b52-45f2-abe8-4af2b3eefe82 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:45.921115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29df2-2b52-45f2-abe8-4af2b3eefe82 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:45.936807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.937522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.938007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.938351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] PATH_INFO: `/v3/projects/435e703327934a33866c051dbd4fa48e/users/c520032ac045435a9e799308af6744be/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.956026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c995c-87f5-4db1-be28-42d7c33615d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:45.956514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c995c-87f5-4db1-be28-42d7c33615d0 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:45.956795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c995c-87f5-4db1-be28-42d7c33615d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:45.957070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c995c-87f5-4db1-be28-42d7c33615d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:45.981726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] RBAC: Authorizing `identity:create_grant(project_id=435e703327934a33866c051dbd4fa48e, user_id=c520032ac045435a9e799308af6744be, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:45.990039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:46.024497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 828/1628] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:45 2022] PUT /identity/v3/projects/435e703327934a33866c051dbd4fa48e/users/c520032ac045435a9e799308af6744be/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:46.027872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:46.080652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c995c-87f5-4db1-be28-42d7c33615d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:46.084622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c995c-87f5-4db1-be28-42d7c33615d0 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:46.181435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 801/1629] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:46.183421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77684271-ee38-419d-af7f-a906fdc2e95d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.183774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77684271-ee38-419d-af7f-a906fdc2e95d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.184063 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77684271-ee38-419d-af7f-a906fdc2e95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.216687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-77684271-ee38-419d-af7f-a906fdc2e95d None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:46.218994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-77684271-ee38-419d-af7f-a906fdc2e95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:46.224125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb42e97-e19d-47df-8294-f71fb334d70d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:46.243305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:46.244241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.244530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.251762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] PATH_INFO: `/v3/projects/d28a2924960b484b8a80e4d6631c1119/users/ee1dec4ce04d43868d1c4318ec71ab62/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.257803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28a2924960b484b8a80e4d6631c1119, user_id=ee1dec4ce04d43868d1c4318ec71ab62, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:46.260670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:46.308027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 829/1630] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:46 2022] PUT /identity/v3/projects/d28a2924960b484b8a80e4d6631c1119/users/ee1dec4ce04d43868d1c4318ec71ab62/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:46.308027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:46.405057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 802/1631] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Jul 21 22:32:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:46.406522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087c98a-a498-401d-8b20-22fcc72acf37 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.406811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087c98a-a498-401d-8b20-22fcc72acf37 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.407108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087c98a-a498-401d-8b20-22fcc72acf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.449345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d087c98a-a498-401d-8b20-22fcc72acf37 None None] MFA Rules not processed for user `ee1dec4ce04d43868d1c4318ec71ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:46.451680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d087c98a-a498-401d-8b20-22fcc72acf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:46.456709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee580f8-47d3-408b-9234-4d1b2ed8ae61 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:46.477056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50b21b-d868-4651-982d-ad32c786b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:46.477918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50b21b-d868-4651-982d-ad32c786b839 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.478269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50b21b-d868-4651-982d-ad32c786b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.478612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50b21b-d868-4651-982d-ad32c786b839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.479456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50b21b-d868-4651-982d-ad32c786b839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:46.481168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50b21b-d868-4651-982d-ad32c786b839 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:46.522105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 830/1632] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:46.523247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff5e24-daf7-4160-b923-43943f79db73 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.524418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:46.525250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff5e24-daf7-4160-b923-43943f79db73 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.525381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff5e24-daf7-4160-b923-43943f79db73 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.534709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 831/1633] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jul 21 22:32:46 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:32:46.536158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff5e24-daf7-4160-b923-43943f79db73 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:46.565417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff5e24-daf7-4160-b923-43943f79db73 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:46.571724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c151ce8-9182-49a9-9ca6-9b678c88c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:46.572714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c151ce8-9182-49a9-9ca6-9b678c88c336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.573075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c151ce8-9182-49a9-9ca6-9b678c88c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.573417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c151ce8-9182-49a9-9ca6-9b678c88c336 admin admin] PATH_INFO: `/v3/projects/435e703327934a33866c051dbd4fa48e/users/c520032ac045435a9e799308af6744be/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.578802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c151ce8-9182-49a9-9ca6-9b678c88c336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=435e703327934a33866c051dbd4fa48e, user_id=c520032ac045435a9e799308af6744be, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:46.581608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c151ce8-9182-49a9-9ca6-9b678c88c336 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:46.615286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 832/1634] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:46 2022] PUT /identity/v3/projects/435e703327934a33866c051dbd4fa48e/users/c520032ac045435a9e799308af6744be/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:46.616863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463e74-8e94-48ff-82ad-42d4abcf3e5a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.617207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463e74-8e94-48ff-82ad-42d4abcf3e5a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.622884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463e74-8e94-48ff-82ad-42d4abcf3e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.660805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5e463e74-8e94-48ff-82ad-42d4abcf3e5a None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:46.662318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5e463e74-8e94-48ff-82ad-42d4abcf3e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:46.668293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 803/1635] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:46.671201 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.671505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.671788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.722954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] MFA Rules not processed for user `c520032ac045435a9e799308af6744be`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:46.728727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:46.746387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:46.854721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 804/1636] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:46 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:46.854721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:46.856231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 833/1637] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:32:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:46.859417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a4843-5048-4675-98cf-b1905b204dff None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.863682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a4843-5048-4675-98cf-b1905b204dff None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.864187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a4843-5048-4675-98cf-b1905b204dff None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:46.866118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 834/1638] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jul 21 22:32:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:32:46.867440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a4843-5048-4675-98cf-b1905b204dff None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:46.963372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08d001-ac4b-4763-ace6-87b333e16ddf None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:46.971732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:46.972547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:46.973120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:46.973417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:47.001655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a4843-5048-4675-98cf-b1905b204dff None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:47.010353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cfebf-bb82-43d8-97d1-5c133f864dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:47.011376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cfebf-bb82-43d8-97d1-5c133f864dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:47.011990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cfebf-bb82-43d8-97d1-5c133f864dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:47.012439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cfebf-bb82-43d8-97d1-5c133f864dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:47.037807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cfebf-bb82-43d8-97d1-5c133f864dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:47.043368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cfebf-bb82-43d8-97d1-5c133f864dc2 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:47.070645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 835/1639] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:46 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:47.072309 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:47.072877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:47.073174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:47.104211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:47.105963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:47.106713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:47.117324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 836/1640] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:32:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:47.117603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:47.120571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:47.133348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:47.139159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 805/1641] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:47.156212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209fdc2-4ed4-4a6e-af4e-ea905ecdca03 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:47.164029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:47.164794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:47.165149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:47.165775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:47.194241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:47.201166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:47.211803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:47.224736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997af91-f81d-4707-9d94-5a10c274434d service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:47.229596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:47.230318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:47.230742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:47.231157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:47.243461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 837/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:32:47.263104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:47.265688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:47.319651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 806/1643] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:47.591644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:47.607416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3411-e61f-4638-aff9-e232b720d59e service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:47.617313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:47.618001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:47.618452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:47.618736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:47.767677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:47.769681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:47.806989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 838/1644] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:47 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:48.100032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:48.116505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a77d9c-50e0-4522-a60f-16f5b39a9022 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:48.139612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2339c68-f4d9-4db8-8ff5-cca2d6ded3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:48.140262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2339c68-f4d9-4db8-8ff5-cca2d6ded3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:48.140644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2339c68-f4d9-4db8-8ff5-cca2d6ded3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:48.140933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2339c68-f4d9-4db8-8ff5-cca2d6ded3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:48.189822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:48.304995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:48.315765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2339c68-f4d9-4db8-8ff5-cca2d6ded3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:48.319727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2339c68-f4d9-4db8-8ff5-cca2d6ded3f4 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:48.328472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ecc34-146f-4463-8e4b-c0d1cd1cac3b service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:48.337700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:48.342654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:48.342950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:48.344196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] PATH_INFO: `/v3/users/2a4393720e844b998ad3d309c06b22f9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:48.346109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 807/1645] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:48.346851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4393720e844b998ad3d309c06b22f9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:48.348853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:48.437802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4393720e844b998ad3d309c06b22f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:48.453911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4393720e844b998ad3d309c06b22f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:48.476486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4393720e844b998ad3d309c06b22f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:48.477330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4393720e844b998ad3d309c06b22f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:48.491591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4393720e844b998ad3d309c06b22f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:48.501204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 839/1646] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:32:48 2022] DELETE /identity/v3/users/2a4393720e844b998ad3d309c06b22f9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:32:48.505702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:48.506051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:48.506329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:48.532180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:48.533882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:48.744412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 808/1647] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:48.756169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:48.758352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:48.835504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e4c37-31b6-4d72-9ce1-b3c17b25d26b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:48.841039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc298adf-7860-4929-a671-23d1f0333d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:48.841855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc298adf-7860-4929-a671-23d1f0333d1d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:48.876280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc298adf-7860-4929-a671-23d1f0333d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:48.876280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc298adf-7860-4929-a671-23d1f0333d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:48.876280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc298adf-7860-4929-a671-23d1f0333d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:48.876280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc298adf-7860-4929-a671-23d1f0333d1d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:48.899088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 840/1648] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:32:49.327986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:49.329474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:49.352699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfa9b-1073-4a47-9618-6c646f40c25e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:49.362396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:49.363679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:49.364114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:49.364384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] PATH_INFO: `/v3/projects/11c72a831a934111b1c89a87f9bc81de` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:49.367455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c72a831a934111b1c89a87f9bc81de)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:49.369730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:49.437447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c72a831a934111b1c89a87f9bc81de', 'request_id': 'req-6720c621-c040-412f-8533-5c8de9d42a66'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:49.450913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c72a831a934111b1c89a87f9bc81de', 'request_id': 'req-6720c621-c040-412f-8533-5c8de9d42a66'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:49.459874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c72a831a934111b1c89a87f9bc81de', 'request_id': 'req-6720c621-c040-412f-8533-5c8de9d42a66'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:49.489971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] The token cache is being invalidate because project 11c72a831a934111b1c89a87f9bc81de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:32:49.490279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:49.490966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6720c621-c040-412f-8533-5c8de9d42a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:49.492976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 809/1649] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:32:49 2022] DELETE /identity/v3/projects/11c72a831a934111b1c89a87f9bc81de => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:32:49.497704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:49.498548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:49.498548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:49.548835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:49.551524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:49.831347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:49.832041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:49.832478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:49.834927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 841/1650] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:49.846807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:49.873644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:49.880394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:49.887010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 810/1651] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:32:49.894166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:49.935315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2f24f-2570-44c1-915e-ec8c04b51af2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:49.956649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:49.957516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:49.957806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:49.958089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:49.958872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:49.959217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:49.964019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:49.970097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:50.018396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 842/1652] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:32:50.025217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:50.043268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac27f0-e632-4084-a1e3-2397558ca206 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:50.055048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:50.056810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:50.057108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:50.058771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:50.062368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:50.067572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:50.164031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 811/1653] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:50 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:50.168104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:50.194771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9f0ec-c2cd-44ef-9deb-c1e64e5fc86b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:50.418368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:50.423570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:50.424016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:50.424364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:50.424929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:50.427196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:50.484743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 843/1654] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:50 2022] POST /identity/v3/users => generated 303 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:50.488596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:50.515586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d3f8c-512d-464c-92d8-1e402d4156e0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:50.540596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:50.543330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:50.547236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:50.551691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:50.556846 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:50.562335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:50.617874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 812/1655] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:50.621632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:50.653324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b7319-81f9-41a2-82db-03a0345e0029 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:50.676960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d512805-40aa-4413-bf69-4c55405e63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:50.677949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d512805-40aa-4413-bf69-4c55405e63d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:50.678575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d512805-40aa-4413-bf69-4c55405e63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:50.679026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d512805-40aa-4413-bf69-4c55405e63d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:50.680053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d512805-40aa-4413-bf69-4c55405e63d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:50.682873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d512805-40aa-4413-bf69-4c55405e63d2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:50.727496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 844/1656] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:50.727883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:50.748021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fa0ab-b66d-4337-8f0a-08767172d95f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:50.763211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:50.765606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:50.765944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:50.766228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] PATH_INFO: `/v3/projects/0b58f1d1bfdd46d596de0d039d864b78/users/13b3c08832614826abed38b94434e522/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:50.815362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b58f1d1bfdd46d596de0d039d864b78, user_id=13b3c08832614826abed38b94434e522, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:50.819109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:50.847965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 813/1657] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:50 2022] PUT /identity/v3/projects/0b58f1d1bfdd46d596de0d039d864b78/users/13b3c08832614826abed38b94434e522/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:32:50.859488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:50.860116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:50.860683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:50.970503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] MFA Rules not processed for user `13b3c08832614826abed38b94434e522`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:50.974155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:51.259781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 845/1658] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:51.264362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:51.426687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbbdf-a8e5-4861-88a5-605a184253c3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:51.460967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:51.460967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:51.460967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:51.460967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:51.471440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:51.483784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:51.589402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 814/1659] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:51 2022] POST /identity/v3/projects => generated 403 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:32:51.590738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:51.607349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5257c7f-028f-4808-acae-3caf993bca56 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:51.781076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:51.784042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:51.784555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:51.785214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:51.786175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:51.788840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:51.810485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:51.836378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 846/1660] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:32:51 2022] POST /identity/v3/users => generated 308 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:32:51.838245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:51.853792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0ed5-277e-455a-a5ba-7f135fee3fd4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:51.868183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:51.869264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:51.869654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:51.869933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:51.870782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:51.874190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:51.900064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 815/1661] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:32:51.901726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:51.924334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495738-5ccb-4557-a78b-449f98b5ebaa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:51.930176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:51.931197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:51.931484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:51.931762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] PATH_INFO: `/v3/projects/eb23f39c548f4aa199db44052ab69ef3/users/be45455276dc45548f4ebe83d7440681/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:51.975471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb23f39c548f4aa199db44052ab69ef3, user_id=be45455276dc45548f4ebe83d7440681, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:51.978595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:52.009171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 847/1662] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:51 2022] PUT /identity/v3/projects/eb23f39c548f4aa199db44052ab69ef3/users/be45455276dc45548f4ebe83d7440681/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:52.014686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:52.034141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c52220-0543-4968-a192-ffe48e29b5a1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:52.172666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31ba3d-5247-4a48-90b1-b6026f94bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:52.173535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31ba3d-5247-4a48-90b1-b6026f94bb66 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:52.173868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31ba3d-5247-4a48-90b1-b6026f94bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:52.174229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31ba3d-5247-4a48-90b1-b6026f94bb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:52.175180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31ba3d-5247-4a48-90b1-b6026f94bb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:52.177007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31ba3d-5247-4a48-90b1-b6026f94bb66 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:52.196609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 816/1663] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:32:52.198678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:52.211986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e96186-1222-427f-8f11-c6bd3e6e7025 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:52.219252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:52.220275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:52.220562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:52.221315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] PATH_INFO: `/v3/projects/eb23f39c548f4aa199db44052ab69ef3/users/be45455276dc45548f4ebe83d7440681/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:52.224231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb23f39c548f4aa199db44052ab69ef3, user_id=be45455276dc45548f4ebe83d7440681, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:52.228209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:52.241402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 848/1664] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:32:52 2022] PUT /identity/v3/projects/eb23f39c548f4aa199db44052ab69ef3/users/be45455276dc45548f4ebe83d7440681/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:52.253626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:52.254402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:52.254515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:52.311841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:52.328297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] MFA Rules not processed for user `be45455276dc45548f4ebe83d7440681`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:52.330016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:52.577933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 817/1665] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:52.646264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:52.650445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:52.768988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2c88-dcfa-4c39-a54f-85440c5d3a8c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:52.781299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66b6fc-8d28-44a2-9acc-32a68a322ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:52.782251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66b6fc-8d28-44a2-9acc-32a68a322ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:52.782409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66b6fc-8d28-44a2-9acc-32a68a322ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:52.782651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66b6fc-8d28-44a2-9acc-32a68a322ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:52.805583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66b6fc-8d28-44a2-9acc-32a68a322ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:52.812517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66b6fc-8d28-44a2-9acc-32a68a322ab1 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:52.853866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 849/1666] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:54.291019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:54.467451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86d2e-2bdd-4272-b234-bef244ad614e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:54.478647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:54.480662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:54.480662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:54.480854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] PATH_INFO: `/v3/users/13b3c08832614826abed38b94434e522` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:54.485255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b3c08832614826abed38b94434e522)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:54.488364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:32:54.488364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 21 22:32:54.488364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: Jul 21 22:32:54.494024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:54.676440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b3c08832614826abed38b94434e522'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:54.705367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b3c08832614826abed38b94434e522'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:54.706770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b3c08832614826abed38b94434e522'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:54.745800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b3c08832614826abed38b94434e522'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:54.846533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b3c08832614826abed38b94434e522'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:54.865681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:54.871216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:54.873613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:54.879596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 818/1667] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:32:54 2022] DELETE /identity/v3/users/13b3c08832614826abed38b94434e522 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:54.922317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:54.924602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:55.130088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 850/1668] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:55.141808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:55.145365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:55.209634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:55.348613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fe7f-60f1-4eec-a3e9-7266682f3c44 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:55.371934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:55.374164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:55.374488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:55.380718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:55.401345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:55.403766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:55.445771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 819/1669] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:55.915917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:55.920819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:55.962984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a425c0-28ed-4442-b5af-3177b46d4c3e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:55.980384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:55.981469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:55.981864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:55.982140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] PATH_INFO: `/v3/projects/0b58f1d1bfdd46d596de0d039d864b78` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:55.983556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b58f1d1bfdd46d596de0d039d864b78)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:55.984613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:32:55.984613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 21 22:32:55.984613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: Jul 21 22:32:55.985887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:56.165610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b58f1d1bfdd46d596de0d039d864b78', 'request_id': 'req-7198f0cb-b244-4def-9f0f-7803a1087230'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.199614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b58f1d1bfdd46d596de0d039d864b78', 'request_id': 'req-7198f0cb-b244-4def-9f0f-7803a1087230'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.212813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:56.222127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b58f1d1bfdd46d596de0d039d864b78', 'request_id': 'req-7198f0cb-b244-4def-9f0f-7803a1087230'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.240512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] The token cache is being invalidate because project 0b58f1d1bfdd46d596de0d039d864b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:32:56.240871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.242000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.244167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 851/1670] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:32:55 2022] DELETE /identity/v3/projects/0b58f1d1bfdd46d596de0d039d864b78 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:56.248870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:56.263854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8c0a-4732-4279-a4de-0392bd710722 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:56.369028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:56.373490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:56.374179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:56.375113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] PATH_INFO: `/v3/users/be45455276dc45548f4ebe83d7440681` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:56.380504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be45455276dc45548f4ebe83d7440681)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:56.386237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:56.510291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be45455276dc45548f4ebe83d7440681'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.534615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be45455276dc45548f4ebe83d7440681'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.536294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be45455276dc45548f4ebe83d7440681'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.557241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be45455276dc45548f4ebe83d7440681'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.593251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-34c814b6-c069-40c2-a569-a6daf690bc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be45455276dc45548f4ebe83d7440681'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:56.603163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 820/1671] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:32:56 2022] DELETE /identity/v3/users/be45455276dc45548f4ebe83d7440681 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:32:59.047548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:59.049089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:59.085900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198f0cb-b244-4def-9f0f-7803a1087230 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:59.318179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:59.319454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:59.319876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:59.320414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] PATH_INFO: `/v3/projects/eb23f39c548f4aa199db44052ab69ef3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:59.326894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb23f39c548f4aa199db44052ab69ef3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:59.329371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:59.430529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb23f39c548f4aa199db44052ab69ef3', 'request_id': 'req-3481e790-fcbc-47a4-b244-0d7c1205172c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:59.439160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb23f39c548f4aa199db44052ab69ef3', 'request_id': 'req-3481e790-fcbc-47a4-b244-0d7c1205172c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:59.457043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb23f39c548f4aa199db44052ab69ef3', 'request_id': 'req-3481e790-fcbc-47a4-b244-0d7c1205172c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:59.494525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] The token cache is being invalidate because project eb23f39c548f4aa199db44052ab69ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:32:59.494903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:59.495964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3481e790-fcbc-47a4-b244-0d7c1205172c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:32:59.499181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 852/1672] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:32:59 2022] DELETE /identity/v3/projects/eb23f39c548f4aa199db44052ab69ef3 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:32:59.504436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:59.506698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:59.506918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:59.541074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:59.544940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:59.595314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:59.675800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 821/1673] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:59.686535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:59.690478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:59.691300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:59.742191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:59.750270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:59.790520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 853/1674] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:59.792066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:59.821462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8f2d-2c5b-42e1-8a09-eb3300ecd0f0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:59.837516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:59.838576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:59.839814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:59.840854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:59.844742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:32:59.845316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:59.848287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:59.885668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 822/1675] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:32:59.890827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:59.906769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b442c-ba2c-4be3-918a-ef5704cc5775 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:59.920286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:59.925982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:59.926289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:59.926573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:59.928102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:59.932268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:00.010505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 854/1676] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:32:59 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:00.011781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:00.023978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:00.025847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:00.039566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698cb5-4374-4d29-8739-7e50d75c59e3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:00.210490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:00.232004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:00.232884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.233176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.233459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.234303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:00.236634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:00.261096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e314a-e888-40ea-96b0-e05cd527caeb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:00.273960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:00.275235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.275762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.276245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] PATH_INFO: `/v3/users/4915c900a2bc460283d237e6c8005117` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.279907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4915c900a2bc460283d237e6c8005117)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:00.282500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:00.327234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 823/1677] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:00 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:00.328375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:00.375771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550c84-0947-4fbc-a6d2-019fc8c78b5d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:00.384888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bd0fd-b13d-4a4c-9d27-8f3560d6f092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:00.385606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd0fd-b13d-4a4c-9d27-8f3560d6f092 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.386239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd0fd-b13d-4a4c-9d27-8f3560d6f092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.386239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bd0fd-b13d-4a4c-9d27-8f3560d6f092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.387773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bd0fd-b13d-4a4c-9d27-8f3560d6f092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:00.389506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bd0fd-b13d-4a4c-9d27-8f3560d6f092 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:00.417993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 824/1678] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:00.436346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c388bef-b6ad-4e55-9efb-99572ab2d71e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.436540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c388bef-b6ad-4e55-9efb-99572ab2d71e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.436828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c388bef-b6ad-4e55-9efb-99572ab2d71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.487595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4915c900a2bc460283d237e6c8005117'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:00.529654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4915c900a2bc460283d237e6c8005117'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:00.566626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4915c900a2bc460283d237e6c8005117'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:00.566626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4915c900a2bc460283d237e6c8005117'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:00.572337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8c388bef-b6ad-4e55-9efb-99572ab2d71e None None] MFA Rules not processed for user `cf3fd4428e3b46cd9e83e1d32571097b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:00.572337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8c388bef-b6ad-4e55-9efb-99572ab2d71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:00.599774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4915c900a2bc460283d237e6c8005117'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:00.612960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 855/1679] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:00 2022] DELETE /identity/v3/users/4915c900a2bc460283d237e6c8005117 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:00.612960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:00.612960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:00.761085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 825/1680] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:00.764066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.764627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.765179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.804881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:00.806510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:00.868848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b6fd-e77e-49e8-bbff-0ca6d2c9dcb3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:00.870687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:00.885697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5217a51-8879-4f83-b947-d79e4f608b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:00.888778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5217a51-8879-4f83-b947-d79e4f608b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.889314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5217a51-8879-4f83-b947-d79e4f608b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.891144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5217a51-8879-4f83-b947-d79e4f608b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.892249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5217a51-8879-4f83-b947-d79e4f608b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:00.897468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5217a51-8879-4f83-b947-d79e4f608b34 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:00.918734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 856/1681] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:00.924125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:00.924980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:00.927823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:00.963975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] MFA Rules not processed for user `cf3fd4428e3b46cd9e83e1d32571097b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:00.965852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:00.969791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 826/1682] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:00.972821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:00.997602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179b41e-addd-4745-835d-ff7cf1af7710 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:01.016480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:01.031979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:01.031979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:01.032279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] PATH_INFO: `/v3/projects/e6b2d2c319f6431eaadda5e2f3ebda76/users/4374dfef6a6a4fdbab3b3a702c562d2b/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:01.092425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 857/1683] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:01.095908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:01.099157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:01.100734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b2d2c319f6431eaadda5e2f3ebda76, user_id=4374dfef6a6a4fdbab3b3a702c562d2b, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:01.106631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:01.121732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:01.157417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 827/1684] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:00 2022] PUT /identity/v3/projects/e6b2d2c319f6431eaadda5e2f3ebda76/users/4374dfef6a6a4fdbab3b3a702c562d2b/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:01.158360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:01.161160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:01.352555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a55d58-402a-4bb1-9d3b-97ac637525e3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:01.396270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9e1f-770e-4db6-8c9c-74c36ec75c1f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:01.414926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048080a-da49-4d2a-b98e-4daa6ff80074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:01.415636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048080a-da49-4d2a-b98e-4daa6ff80074 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:01.415922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048080a-da49-4d2a-b98e-4daa6ff80074 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:01.416207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048080a-da49-4d2a-b98e-4daa6ff80074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:01.508008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:01.509727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:01.510058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:01.510462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:01.602325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048080a-da49-4d2a-b98e-4daa6ff80074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:01.608809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048080a-da49-4d2a-b98e-4daa6ff80074 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:01.624657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 828/1685] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:01.628165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:01.629521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:01.639900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:01.671402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:01.673169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:01.693642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] MFA Rules not processed for user `4374dfef6a6a4fdbab3b3a702c562d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:01.708664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 858/1686] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:01.719393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:01.807247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:01.870750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 829/1687] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:01.877768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:01.900699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab5ab1-a0c6-40d0-89a2-1111e7883fd9 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:01.908996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:01.910063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:01.911089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:01.911715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:01.912834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:01.915493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:01.958628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 859/1688] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:01 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:01.962836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:01.974166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068be925-b784-4f9b-ae87-a232c5817a47 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:02.108670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:02.110172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.110787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.111253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.112181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:02.117397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:02.161038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 830/1689] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:01 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:02.175163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:02.197063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599dfbf-5e48-4507-aace-cd4e7dc28f4b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:02.214246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:02.215371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.215805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.216319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.218793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:02.225101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:02.269978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 860/1690] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:02.271400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:02.289298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d09c7b-2fa0-4636-9306-581be09524f4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:02.304607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:02.310205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.310728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.311211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] PATH_INFO: `/v3/projects/dae0e55ceecd4c6bac417e394ce17bf0/users/c37a60347744469997dce83506a74e08/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.370872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae0e55ceecd4c6bac417e394ce17bf0, user_id=c37a60347744469997dce83506a74e08, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:02.375620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:02.407351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 831/1691] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:02 2022] PUT /identity/v3/projects/dae0e55ceecd4c6bac417e394ce17bf0/users/c37a60347744469997dce83506a74e08/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:02.411220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:02.429568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75317859-3a9c-401b-bd4b-d11c895f9b87 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:02.556744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:02.621552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cfc2b-8f07-4093-a67e-2872220ebef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:02.623904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cfc2b-8f07-4093-a67e-2872220ebef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.624423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cfc2b-8f07-4093-a67e-2872220ebef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.625184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cfc2b-8f07-4093-a67e-2872220ebef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.627483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cfc2b-8f07-4093-a67e-2872220ebef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:02.633024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cfc2b-8f07-4093-a67e-2872220ebef9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:02.669208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 861/1692] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:02.670547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:02.693711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16b725-945f-483c-8289-0a46d4a9907a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:02.714712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e3c7e-06d1-4e8c-9807-2941465ff6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:02.716514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e3c7e-06d1-4e8c-9807-2941465ff6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.719627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e3c7e-06d1-4e8c-9807-2941465ff6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.721024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e3c7e-06d1-4e8c-9807-2941465ff6ba admin admin] PATH_INFO: `/v3/projects/dae0e55ceecd4c6bac417e394ce17bf0/users/c37a60347744469997dce83506a74e08/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.726207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e3c7e-06d1-4e8c-9807-2941465ff6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae0e55ceecd4c6bac417e394ce17bf0, user_id=c37a60347744469997dce83506a74e08, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:02.735147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8f4a2-fe8a-44f3-8587-aa0cf06b5e12 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.735147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8f4a2-fe8a-44f3-8587-aa0cf06b5e12 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.735583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8f4a2-fe8a-44f3-8587-aa0cf06b5e12 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.736083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e3c7e-06d1-4e8c-9807-2941465ff6ba admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:02.739003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 862/1693] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jul 21 22:33:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:02.749415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.749709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.749993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.779768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 832/1694] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:02 2022] PUT /identity/v3/projects/dae0e55ceecd4c6bac417e394ce17bf0/users/c37a60347744469997dce83506a74e08/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:02.782822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792ca2-47f5-4497-b718-dd8f2637cd31 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:02.783353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792ca2-47f5-4497-b718-dd8f2637cd31 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:02.783620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792ca2-47f5-4497-b718-dd8f2637cd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:02.868040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b7792ca2-47f5-4497-b718-dd8f2637cd31 None None] MFA Rules not processed for user `c37a60347744469997dce83506a74e08`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:02.869554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b7792ca2-47f5-4497-b718-dd8f2637cd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:02.888364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] MFA Rules not processed for user `7f5dceea71d1434796a105ea67773cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:02.899886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:03.089990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 833/1695] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:03.089990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:03.089990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:03.089990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:03.116914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 863/1696] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:33:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:03.119765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:03.128128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:03.148295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:03.164101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] MFA Rules not processed for user `2c07a46f581b4bbb97ad99005471ebe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:03.167692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:03.364079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 834/1697] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:03.368085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d7d09-5a4c-451a-bc29-a251476ddfe0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:03.375151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:03.376933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6483b-27c3-4bc0-b55e-b2ce6ca63f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:03.378509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6483b-27c3-4bc0-b55e-b2ce6ca63f36 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:03.378509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6483b-27c3-4bc0-b55e-b2ce6ca63f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:03.378797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6483b-27c3-4bc0-b55e-b2ce6ca63f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:03.398941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6483b-27c3-4bc0-b55e-b2ce6ca63f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:03.405701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6483b-27c3-4bc0-b55e-b2ce6ca63f36 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:03.406024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac792b49-cad8-45c1-a991-8e0cb45914a6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:03.413543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:03.414220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:03.414584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:03.415019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:03.427940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 864/1698] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:03.434995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:03.436974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:03.438004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:03.443998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:03.451764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:03.489279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] MFA Rules not processed for user `2c07a46f581b4bbb97ad99005471ebe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:03.492659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 835/1699] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:03.495701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:03.527089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 865/1700] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:03.542493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:03.544125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:03.760045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c56cf-fe3e-4179-960f-42a6c9de7209 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:03.776293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569738ba-7919-4a9f-b4a7-6fbc9c01476f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:03.780279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569738ba-7919-4a9f-b4a7-6fbc9c01476f service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:03.780714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569738ba-7919-4a9f-b4a7-6fbc9c01476f service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:03.781577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569738ba-7919-4a9f-b4a7-6fbc9c01476f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:03.804336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569738ba-7919-4a9f-b4a7-6fbc9c01476f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:03.812957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569738ba-7919-4a9f-b4a7-6fbc9c01476f service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:03.821845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:03.823423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:03.867952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 836/1701] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:03.992206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b3d3-931a-4d57-8a19-24f7a2b90782 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:04.004197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:04.005099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.005387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.019314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] PATH_INFO: `/v3/projects/376889a38a46462d9ba21b0f94e45a1a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.021093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=376889a38a46462d9ba21b0f94e45a1a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:04.024429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:04.062274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:04.192935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376889a38a46462d9ba21b0f94e45a1a', 'request_id': 'req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:04.204489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376889a38a46462d9ba21b0f94e45a1a', 'request_id': 'req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:04.221122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376889a38a46462d9ba21b0f94e45a1a', 'request_id': 'req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:04.249980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] The token cache is being invalidate because project 376889a38a46462d9ba21b0f94e45a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:04.250566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:04.251987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62351d3f-e6a7-4d99-9193-c5ea6b084ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:04.256503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 866/1702] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:03 2022] DELETE /identity/v3/projects/376889a38a46462d9ba21b0f94e45a1a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:04.262385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba2825-eaf8-4842-9358-3b5d0dc4fafe None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.262881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba2825-eaf8-4842-9358-3b5d0dc4fafe None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.263364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba2825-eaf8-4842-9358-3b5d0dc4fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.295104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-10ba2825-eaf8-4842-9358-3b5d0dc4fafe None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:04.297995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-10ba2825-eaf8-4842-9358-3b5d0dc4fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:04.545375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 837/1703] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:04.548005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.548429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.548530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.595929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:04.597228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:04.615047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c643-33f5-4572-8ccc-f9f60c84f1da None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.615849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c643-33f5-4572-8ccc-f9f60c84f1da None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.616213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c643-33f5-4572-8ccc-f9f60c84f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.649450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 867/1704] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:04.651091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:04.686592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009ee3c-f0d9-4108-b072-0fc891bea762 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:04.718753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:04.720095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.720493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.721188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.724891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:04.727149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:04.728593 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:04.736381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2711d-9345-425e-aa07-bb0bb8f5ef79 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:04.744167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4163c643-33f5-4572-8ccc-f9f60c84f1da None None] MFA Rules not processed for user `a6146c8083d04171863a4b603fd6a46c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:04.745822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4163c643-33f5-4572-8ccc-f9f60c84f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:04.805306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 868/1705] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:04.808365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092255f-4e69-4688-a2c0-4d931860f609 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.808365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092255f-4e69-4688-a2c0-4d931860f609 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.808503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092255f-4e69-4688-a2c0-4d931860f609 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.825606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 869/1706] 10.222.0.58 () {54 vars in 961 bytes} [Thu Jul 21 22:33:04 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:04.826574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092255f-4e69-4688-a2c0-4d931860f609 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:04.881176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092255f-4e69-4688-a2c0-4d931860f609 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:04.906608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd8db2-40b1-4ae8-8971-6bce025bc7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:04.907953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd8db2-40b1-4ae8-8971-6bce025bc7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.908239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd8db2-40b1-4ae8-8971-6bce025bc7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.908513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd8db2-40b1-4ae8-8971-6bce025bc7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:04.909291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd8db2-40b1-4ae8-8971-6bce025bc7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:04.914387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd8db2-40b1-4ae8-8971-6bce025bc7b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:04.996014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 870/1707] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:04 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:04.997981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433c337-13fd-4c82-9785-7a719dfe63bf None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:04.998274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433c337-13fd-4c82-9785-7a719dfe63bf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:04.998550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433c337-13fd-4c82-9785-7a719dfe63bf None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.005293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 871/1708] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jul 21 22:33:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:05.007025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1b2b1-8e50-42dd-8010-c008e97e88b2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.007025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1b2b1-8e50-42dd-8010-c008e97e88b2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.007249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1b2b1-8e50-42dd-8010-c008e97e88b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.019986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 838/1709] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:05.024154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.024648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.029567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.053171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d2f1b2b1-8e50-42dd-8010-c008e97e88b2 None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:05.055538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d2f1b2b1-8e50-42dd-8010-c008e97e88b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:05.084410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] MFA Rules not processed for user `cf3fd4428e3b46cd9e83e1d32571097b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:05.100504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:05.256394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 839/1710] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:05.257656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:05.257881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 872/1711] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Jul 21 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:05.267763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.267985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.268318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.345914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] MFA Rules not processed for user `7f5dceea71d1434796a105ea67773cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:05.348079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:05.422752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a9627-9d44-4a99-afd0-4e0294b127ff None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:05.436175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:05.438522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.439120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.440295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.444423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:05.447126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:05.470982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:05.487731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 873/1712] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:05.488677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:05.491642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:05.515088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 840/1713] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:05 2022] POST /identity/v3/users => generated 284 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:05.515088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:05.530901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0660-84c0-400c-8ea5-25447a1f38cd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:05.553535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:05.554338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.554733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.555302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.557600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:05.589421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:05.591125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:05.619729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 841/1714] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:05.619729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:05.622190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:05.660462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b414f5-39d1-4b95-9ddf-548f7419b1c0 service placement] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:05.664061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:05.665896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.667919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.669226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.701637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:05.702374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:05.733884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 842/1715] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:05.743332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:05.794520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542dacb-6917-4ad6-806e-1630a0dbf5bf service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:05.819016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:05.819749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.820036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.820362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.828435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:05.835631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:05.858210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb98e49-cf97-4639-8d9c-314043f30422 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:05.869366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:05.889660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:05.894435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.895242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.896926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.911836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 843/1716] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:05.924645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:05.969221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c944-b3e4-46a5-aa5f-639986f29bb3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:05.992605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1316a-afc4-4c71-85a9-6762f8c442cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:05.993432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1316a-afc4-4c71-85a9-6762f8c442cb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:05.993679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1316a-afc4-4c71-85a9-6762f8c442cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:05.993972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1316a-afc4-4c71-85a9-6762f8c442cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:05.994880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1316a-afc4-4c71-85a9-6762f8c442cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:05.998195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1316a-afc4-4c71-85a9-6762f8c442cb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:06.028371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 844/1717] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:06.031353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:06.031825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:06.032257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:06.089249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] MFA Rules not processed for user `1ee22783b3234c4f905fedce7d59f01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:06.090711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:06.093226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:06.095033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:06.107004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 874/1718] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:06.108067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:06.121120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4c397-fe36-4d1e-b3a8-dfd9982e2ddb service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:06.130231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7327-f677-4952-bcc0-469610b610ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:06.131407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7327-f677-4952-bcc0-469610b610ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:06.132171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7327-f677-4952-bcc0-469610b610ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:06.132620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7327-f677-4952-bcc0-469610b610ed admin admin] PATH_INFO: `/v3/projects/626a5085e6e74e9f8fe50acea9e5a6bb/users/639f6db7d86c4aac9963743f5fbab097/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:06.178990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7327-f677-4952-bcc0-469610b610ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=626a5085e6e74e9f8fe50acea9e5a6bb, user_id=639f6db7d86c4aac9963743f5fbab097, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:06.181929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7327-f677-4952-bcc0-469610b610ed admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:06.220001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 875/1719] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:06 2022] PUT /identity/v3/projects/626a5085e6e74e9f8fe50acea9e5a6bb/users/639f6db7d86c4aac9963743f5fbab097/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:06.222701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:06.223151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:06.223543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:06.272963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] MFA Rules not processed for user `639f6db7d86c4aac9963743f5fbab097`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:06.274463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:06.370566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 845/1720] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:06.435027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 876/1721] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:06.440097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:06.477185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00138b50-72b0-4ad6-ba2f-0a79640d733b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:06.754884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:06.758940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:06.760617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:06.760966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:06.766424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:06.768590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:06.874229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:06.876245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 846/1722] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:06 2022] POST /identity/v3/projects => generated 369 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:06.882570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:06.901014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630985b1-c90d-40ff-bdf8-e5e738d18431 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:07.112884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:07.115570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:07.119850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:07.120746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:07.121036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:07.123157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:07.125763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:07.131824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:07.186089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 877/1723] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:06 2022] POST /identity/v3/users => generated 291 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:07.188235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:07.232045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7b393-7c1c-446f-acaa-cf7c97ce9318 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:07.270937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:07.273528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:07.274632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:07.278680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:07.282357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:07.289392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:07.339745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 878/1724] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:07.343985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:07.373041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba72a5-d0fc-4a37-87fd-93555739e481 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:07.405574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:07.419516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:07.419650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:07.420027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:07.421720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] PATH_INFO: `/v3/projects/1ccb9efac24b421287425063cc965e93/users/2acdb6e81bcd4135bd999d732e122417/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:07.462819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592013d-e5f6-4507-8a13-52c93b4e6ac9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:07.487023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:07.490240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:07.490994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:07.491445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:07.491734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccb9efac24b421287425063cc965e93, user_id=2acdb6e81bcd4135bd999d732e122417, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:07.506004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:07.575284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 879/1725] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:07 2022] PUT /identity/v3/projects/1ccb9efac24b421287425063cc965e93/users/2acdb6e81bcd4135bd999d732e122417/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:07.590499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:07.743547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:07.758388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:07.805548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 847/1726] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:07.912510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbbdc4-2643-4474-b201-dee757e3bf8a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:07.943896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470fe63-50cf-492d-a9f1-892cb171ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:07.944619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470fe63-50cf-492d-a9f1-892cb171ce0f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:07.944962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470fe63-50cf-492d-a9f1-892cb171ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:07.945261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470fe63-50cf-492d-a9f1-892cb171ce0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:07.946010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470fe63-50cf-492d-a9f1-892cb171ce0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:07.947822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470fe63-50cf-492d-a9f1-892cb171ce0f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:07.997678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 880/1727] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:07.998889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:08.019081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be0fd5-ad2a-4316-9002-7ed0b882c7a4 service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:08.042592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:08.049588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:08.052755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:08.053173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] PATH_INFO: `/v3/projects/1ccb9efac24b421287425063cc965e93/users/2acdb6e81bcd4135bd999d732e122417/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:08.063516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccb9efac24b421287425063cc965e93, user_id=2acdb6e81bcd4135bd999d732e122417, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:08.084498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:08.158819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 848/1728] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:07 2022] PUT /identity/v3/projects/1ccb9efac24b421287425063cc965e93/users/2acdb6e81bcd4135bd999d732e122417/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:08.164742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:08.165286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:08.165674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:08.257251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] MFA Rules not processed for user `2acdb6e81bcd4135bd999d732e122417`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:08.258658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:08.385809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:08.389765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:08.412306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:08.486524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 881/1729] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:08 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:08.498368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:08.500260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:08.582459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8e88-8a88-4ac8-801e-616febfc0a9b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:08.595532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:08.596613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:08.597021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:08.597407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] PATH_INFO: `/v3/users/4374dfef6a6a4fdbab3b3a702c562d2b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:08.599704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4374dfef6a6a4fdbab3b3a702c562d2b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:08.601830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:08.659172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d545ef-e453-47af-9fe5-8ca000a5325e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:08.670955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:08.672440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:08.672728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:08.673015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:08.713217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:08.717878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:08.767584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4374dfef6a6a4fdbab3b3a702c562d2b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:08.792289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4374dfef6a6a4fdbab3b3a702c562d2b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:08.794418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4374dfef6a6a4fdbab3b3a702c562d2b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:08.806347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 882/1730] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:08 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:08.821490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4374dfef6a6a4fdbab3b3a702c562d2b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:08.905538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6d49bbab-71a7-4ed3-998f-c4378fef9434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4374dfef6a6a4fdbab3b3a702c562d2b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:08.951404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:08.964639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3f44-cdb5-4178-8b40-9284db7c36eb service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:08.974739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 849/1731] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:08 2022] DELETE /identity/v3/users/4374dfef6a6a4fdbab3b3a702c562d2b => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:08.986377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3d85f-8511-4c87-bbd3-2387e1249138 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:08.987895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3d85f-8511-4c87-bbd3-2387e1249138 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:08.990533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3d85f-8511-4c87-bbd3-2387e1249138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:09.075798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:09.120612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c9b3d85f-8511-4c87-bbd3-2387e1249138 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:09.146713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c9b3d85f-8511-4c87-bbd3-2387e1249138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:09.259569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:09.261189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:09.265506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:09.266094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:09.269816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:09.487902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:09.491244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:09.515092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 883/1732] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:09.556955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 850/1733] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:09.641047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:09.642900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:09.790848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:09.906535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2231f2-9e97-498f-986f-ee4ddf1681ea service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:09.927670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:09.928582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:09.930749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:09.931787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:09.984341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:09.988831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:10.038601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 884/1734] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:10.568178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:10.568721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:10.568721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:10.920903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:10.925752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:10.955820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:10.957282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:11.023906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa419c-4b71-4317-9c96-b5215cab2887 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:11.051998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:11.060193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:11.060491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:11.060782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] PATH_INFO: `/v3/users/3b054be28a9142618371ad0fc1d63510` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:11.069966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b054be28a9142618371ad0fc1d63510)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:11.078491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:11.221747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b054be28a9142618371ad0fc1d63510'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:11.251708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b054be28a9142618371ad0fc1d63510'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:11.309834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b054be28a9142618371ad0fc1d63510'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:11.313873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b054be28a9142618371ad0fc1d63510'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:11.339141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-17048a2a-9de7-4d8b-bf11-38f4081d7ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b054be28a9142618371ad0fc1d63510'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:11.359736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 885/1735] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:10 2022] DELETE /identity/v3/users/3b054be28a9142618371ad0fc1d63510 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:11.362225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:11.362523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:11.362813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:11.415907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:11.417718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:11.481852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 851/1736] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:33:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 915 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:11.527629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 886/1737] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:11.544677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:11.546514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:11.567475 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f0aa-6fe0-4f24-a290-8561acac7338 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:11.733091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:11.734604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:11.735187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:11.735665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:11.757323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:11.759603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:11.776400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 852/1738] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:11.839667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:11.841633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:11.954816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:12.055630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6bbaf-93f9-4708-8882-5165362a6314 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:12.086411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:12.089759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:12.090139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:12.091951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:12.256435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:12.259623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:12.293150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 887/1739] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:33:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:33:12.803758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:12.808251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:12.866477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570c95-7d92-4f25-be14-9696cef0dc53 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:12.879790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:12.881938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:12.882379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:12.882805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] PATH_INFO: `/v3/projects/58e5cbfc341048079874b818fcc619eb` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:12.886329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e5cbfc341048079874b818fcc619eb)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:12.889374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:12.935090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:13.003820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e5cbfc341048079874b818fcc619eb', 'request_id': 'req-49e93282-7941-417d-9bea-8a7465f67704'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.031449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e5cbfc341048079874b818fcc619eb', 'request_id': 'req-49e93282-7941-417d-9bea-8a7465f67704'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.041763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e5cbfc341048079874b818fcc619eb', 'request_id': 'req-49e93282-7941-417d-9bea-8a7465f67704'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.068857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] The token cache is being invalidate because project 58e5cbfc341048079874b818fcc619eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:13.069368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.070823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.077032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 853/1740] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:12 2022] DELETE /identity/v3/projects/58e5cbfc341048079874b818fcc619eb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:13.080580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:13.105074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08affe53-2629-4873-b7f1-a7afecce131f service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:13.311579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:13.312717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:13.313123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:13.313535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] PATH_INFO: `/v3/users/c520032ac045435a9e799308af6744be` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:13.317997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c520032ac045435a9e799308af6744be)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:13.321078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:13.428023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c520032ac045435a9e799308af6744be'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.444914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c520032ac045435a9e799308af6744be'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.475153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c520032ac045435a9e799308af6744be'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.483632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c520032ac045435a9e799308af6744be'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.488806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:13.497578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c520032ac045435a9e799308af6744be'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:13.522667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 888/1741] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:13 2022] DELETE /identity/v3/users/c520032ac045435a9e799308af6744be => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:13.772110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:13.774146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:13.808101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e93282-7941-417d-9bea-8a7465f67704 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:14.008912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:14.010426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:14.010426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:14.010426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] PATH_INFO: `/v3/projects/e6b2d2c319f6431eaadda5e2f3ebda76` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:14.012317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b2d2c319f6431eaadda5e2f3ebda76)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:14.014584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:14.200634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b2d2c319f6431eaadda5e2f3ebda76', 'request_id': 'req-1ac4098d-51a5-448a-856f-4d0335944a46'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.225540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b2d2c319f6431eaadda5e2f3ebda76', 'request_id': 'req-1ac4098d-51a5-448a-856f-4d0335944a46'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.233888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b2d2c319f6431eaadda5e2f3ebda76', 'request_id': 'req-1ac4098d-51a5-448a-856f-4d0335944a46'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.250822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] The token cache is being invalidate because project e6b2d2c319f6431eaadda5e2f3ebda76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:14.251371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.253861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1ac4098d-51a5-448a-856f-4d0335944a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.259972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 854/1742] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:13 2022] DELETE /identity/v3/projects/e6b2d2c319f6431eaadda5e2f3ebda76 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:14.260280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:14.262512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:14.295254 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8b1ac-dc57-4957-9678-a38e95cc8860 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:14.432779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:14.434757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:14.435236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:14.435669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] PATH_INFO: `/v3/users/c37a60347744469997dce83506a74e08` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:14.437737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37a60347744469997dce83506a74e08)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:14.447011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:14.630068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37a60347744469997dce83506a74e08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.666495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37a60347744469997dce83506a74e08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.700311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37a60347744469997dce83506a74e08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.702931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37a60347744469997dce83506a74e08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.734035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37a60347744469997dce83506a74e08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:14.775886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 889/1743] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:14 2022] DELETE /identity/v3/users/c37a60347744469997dce83506a74e08 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:15.716046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:15.716640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:15.716640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:15.755962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] MFA Rules not processed for user `1ee22783b3234c4f905fedce7d59f01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:15.755962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:15.892080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:15.966778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:15.968393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:15.998779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 855/1744] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:16.178094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:16.179588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:16.273513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c5fa-ac69-43fd-b86c-8eae3f2ee27d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:16.284515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:16.285431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.285723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.286062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] PATH_INFO: `/v3/projects/dae0e55ceecd4c6bac417e394ce17bf0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.287328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae0e55ceecd4c6bac417e394ce17bf0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:16.289305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:16.310012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:16.350147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05833f7e-3d89-4678-b93c-d377f4e47798 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:16.385062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae0e55ceecd4c6bac417e394ce17bf0', 'request_id': 'req-07c9ca89-9754-44aa-9a89-69be53aa8be0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.398968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae0e55ceecd4c6bac417e394ce17bf0', 'request_id': 'req-07c9ca89-9754-44aa-9a89-69be53aa8be0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.416040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae0e55ceecd4c6bac417e394ce17bf0', 'request_id': 'req-07c9ca89-9754-44aa-9a89-69be53aa8be0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.456778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] The token cache is being invalidate because project dae0e55ceecd4c6bac417e394ce17bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:16.457162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.458631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-07c9ca89-9754-44aa-9a89-69be53aa8be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.461285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 890/1745] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:15 2022] DELETE /identity/v3/projects/dae0e55ceecd4c6bac417e394ce17bf0 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:16.466526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e127-8204-43f1-b2ef-f5ffa0c198fd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.467734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e127-8204-43f1-b2ef-f5ffa0c198fd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.468023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e127-8204-43f1-b2ef-f5ffa0c198fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.491030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:16.493091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.493596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.494425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] PATH_INFO: `/v3/users/639f6db7d86c4aac9963743f5fbab097` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.496860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639f6db7d86c4aac9963743f5fbab097)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:16.499457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fe05e127-8204-43f1-b2ef-f5ffa0c198fd None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:16.500863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe05e127-8204-43f1-b2ef-f5ffa0c198fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:16.502616 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:16.513289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 891/1746] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:16.524350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.524647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.524991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.565904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:16.579182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:16.607679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 892/1747] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:16.608993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:16.630015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c3ffc-cd83-4bbc-b2d5-64e6228f9267 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:16.639049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:16.640086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.640378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.640664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.641557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:16.641882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:16.643898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:16.664108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639f6db7d86c4aac9963743f5fbab097'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.693247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639f6db7d86c4aac9963743f5fbab097'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.693596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 893/1748] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:16.694511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639f6db7d86c4aac9963743f5fbab097'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.694623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:16.703422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:16.724840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639f6db7d86c4aac9963743f5fbab097'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.736328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce7618-c6b9-4a74-9ff8-97101246af71 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:16.744460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bec2dff6-61ad-4ae3-b8fa-f4e15b9dd406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639f6db7d86c4aac9963743f5fbab097'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:16.750731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:16.751609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.752261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.752688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.754856 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 856/1749] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:16 2022] DELETE /identity/v3/users/639f6db7d86c4aac9963743f5fbab097 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:16.758411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:16.760477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:16.760792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:16.761304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:16.762334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:16.790258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:16.810952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:16.814264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:16.849659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 894/1750] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:16 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:16.856397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.010884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 857/1751] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:17.046219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8f2ef-4612-48c3-ad12-8f2f96b9f84b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:17.049905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.052464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:17.070494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:17.072914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.073584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.074584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.075922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:17.084098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:17.145881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 895/1752] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:16 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:17.150270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.172294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b20ac1-9357-406e-9611-4f79563dc9fb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:17.198242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59041b1d-0363-471d-b290-c0d5c053734a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:17.200209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59041b1d-0363-471d-b290-c0d5c053734a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.200861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59041b1d-0363-471d-b290-c0d5c053734a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.201338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59041b1d-0363-471d-b290-c0d5c053734a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.202726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59041b1d-0363-471d-b290-c0d5c053734a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:17.208592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb2bb7-ad1b-44c5-bb81-996b9d818587 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:17.212463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59041b1d-0363-471d-b290-c0d5c053734a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:17.240210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 896/1753] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:17.242191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eae84-42c9-49ac-bb96-5b919f306775 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.242191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eae84-42c9-49ac-bb96-5b919f306775 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.242365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eae84-42c9-49ac-bb96-5b919f306775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.285217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-aa8eae84-42c9-49ac-bb96-5b919f306775 None None] MFA Rules not processed for user `a6146c8083d04171863a4b603fd6a46c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:17.286483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aa8eae84-42c9-49ac-bb96-5b919f306775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:17.303247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:17.305092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.305330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.305609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.319592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:17.321354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:17.335923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 858/1754] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:17.336759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.348617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf71e2-4fac-424a-981b-e561cfcd25ab service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:17.355012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:17.356645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.356980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.357220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.357946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:17.359635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:17.370868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:17.372992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 859/1755] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:17.377887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.394964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74379398-3b5d-4895-9947-7d1db9b7d964 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:17.402040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:17.403651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.404019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.404371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] PATH_INFO: `/v3/projects/902f1fd8bcdc49849e8ce778bf1bc4ee/users/f2ee5a475b1648699c31bf76a07ee62b/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.425823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] RBAC: Authorizing `identity:create_grant(project_id=902f1fd8bcdc49849e8ce778bf1bc4ee, user_id=f2ee5a475b1648699c31bf76a07ee62b, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:17.430384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:17.448438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 897/1756] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:17.453599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 860/1757] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:17 2022] PUT /identity/v3/projects/902f1fd8bcdc49849e8ce778bf1bc4ee/users/f2ee5a475b1648699c31bf76a07ee62b/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:17.470484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:17.470857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:17.471674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:17.531244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] MFA Rules not processed for user `f2ee5a475b1648699c31bf76a07ee62b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:17.533989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:17.809925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 898/1758] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:17.813619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.984728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:17.990299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:17.992373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:18.094865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019e5f-a736-4aa1-a40e-cc55d6aa288b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:18.112931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd817-fbd3-42da-98de-dc8d2dc5bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:18.113536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd817-fbd3-42da-98de-dc8d2dc5bbec admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:18.113823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd817-fbd3-42da-98de-dc8d2dc5bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:18.114106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd817-fbd3-42da-98de-dc8d2dc5bbec admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:18.114803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfd817-fbd3-42da-98de-dc8d2dc5bbec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:18.127283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfd817-fbd3-42da-98de-dc8d2dc5bbec admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:18.220724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 861/1759] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:17 2022] POST /identity/v3/projects => generated 387 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:18.224836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:18.225270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:18.225687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:18.262858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4000c8-a5cc-467d-a3ac-77b5b3b49156 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:18.280530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] MFA Rules not processed for user `7f5dceea71d1434796a105ea67773cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:18.286235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:18.520910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 862/1760] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jul 21 22:33:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:18.521816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:18.554238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:18.559001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:18.559645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:18.560786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] PATH_INFO: `/v3/projects/435e703327934a33866c051dbd4fa48e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:18.570680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] RBAC: Authorizing `identity:delete_project(project_id=435e703327934a33866c051dbd4fa48e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:18.576531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:18.640935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:18.760973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef6e83-4116-4315-ab98-f69562b48cb0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:18.782095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435e703327934a33866c051dbd4fa48e', 'request_id': 'req-8af43e36-14ee-4e82-9dbe-2264b5de91af'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:18.799236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435e703327934a33866c051dbd4fa48e', 'request_id': 'req-8af43e36-14ee-4e82-9dbe-2264b5de91af'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:18.821942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435e703327934a33866c051dbd4fa48e', 'request_id': 'req-8af43e36-14ee-4e82-9dbe-2264b5de91af'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:18.859906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] The token cache is being invalidate because project 435e703327934a33866c051dbd4fa48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:18.860902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:18.862661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8af43e36-14ee-4e82-9dbe-2264b5de91af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:18.868388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 899/1761] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:17 2022] DELETE /identity/v3/projects/435e703327934a33866c051dbd4fa48e => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:18.874319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837cc8f-0c86-462e-8ff1-5f56f02ba9c0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:18.875006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837cc8f-0c86-462e-8ff1-5f56f02ba9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:18.875864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837cc8f-0c86-462e-8ff1-5f56f02ba9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:18.960935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c837cc8f-0c86-462e-8ff1-5f56f02ba9c0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:18.963625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c837cc8f-0c86-462e-8ff1-5f56f02ba9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:18.981381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:18.982254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:18.982254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:18.982254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:18.983074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:18.993414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 900/1762] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:18.999359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:18.999960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.000552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.029052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.052745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:19.057751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.080993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 901/1763] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:19.082040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.088396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.094212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 863/1764] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:18 2022] POST /identity/v3/users => generated 300 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:19.097927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.113821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:19.123141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7b0c6-aac1-4ebc-96aa-3e01b913569a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.127973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd72bd-12b9-47ce-ae83-8188d8dd9260 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.135749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.137055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.137694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.137993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.141092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:19.141899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.143984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.145065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.146027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.146637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.147040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] PATH_INFO: `/v3/projects/626a5085e6e74e9f8fe50acea9e5a6bb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.150200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=626a5085e6e74e9f8fe50acea9e5a6bb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.154496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.186523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 864/1765] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:19.191361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.193108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.212446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9b278-d47d-41dc-982e-30ab5e9381fc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.218487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.219458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.220241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.220705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.221457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.223192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.237612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 865/1766] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:19.238483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.246870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626a5085e6e74e9f8fe50acea9e5a6bb', 'request_id': 'req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.250744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71603bbe-d54e-45d2-b181-3a9cc5c987d9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.254767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626a5085e6e74e9f8fe50acea9e5a6bb', 'request_id': 'req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.266240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626a5085e6e74e9f8fe50acea9e5a6bb', 'request_id': 'req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.289537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] The token cache is being invalidate because project 626a5085e6e74e9f8fe50acea9e5a6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:19.289815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.290529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.292256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 902/1767] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:19 2022] DELETE /identity/v3/projects/626a5085e6e74e9f8fe50acea9e5a6bb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:19.296366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.297830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.362149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.362782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.363517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.363953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.364773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.367832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.404871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100bef7-5d37-4b0e-862c-d83cdf8ff12c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.405419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 866/1768] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:19 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:19.406403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.410235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.439351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:19.509749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c67fa-e836-438c-8d53-b6d61de0639d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.516886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.518068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.518405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.518744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] PATH_INFO: `/v3/users/2acdb6e81bcd4135bd999d732e122417` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.520464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2acdb6e81bcd4135bd999d732e122417)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.524193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.558024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.559364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.559666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.559949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] PATH_INFO: `/v3/projects/b4a17a8a0eab432b9c3820a7a19f6e23/users/e8c12c57c0704e7f821f34581c17612f/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.601386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a17a8a0eab432b9c3820a7a19f6e23, user_id=e8c12c57c0704e7f821f34581c17612f, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.607647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.614343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2acdb6e81bcd4135bd999d732e122417'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.628778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2acdb6e81bcd4135bd999d732e122417'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.629416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2acdb6e81bcd4135bd999d732e122417'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.644172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 903/1769] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:19 2022] PUT /identity/v3/projects/b4a17a8a0eab432b9c3820a7a19f6e23/users/e8c12c57c0704e7f821f34581c17612f/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:19.650517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.652096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2acdb6e81bcd4135bd999d732e122417'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.654980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:19.660630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.695785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2acdb6e81bcd4135bd999d732e122417'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:19.721429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 867/1770] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:19 2022] DELETE /identity/v3/users/2acdb6e81bcd4135bd999d732e122417 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:19.722342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:19.738736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:19.829953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399403d-60a4-44fb-919b-ee469dd4c643 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.932310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb511-f949-4faf-a349-ff1eed2dcdfa admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:19.946227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:19.954656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:19.955598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:19.956008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:19.956985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:19.959099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:19.992021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 868/1771] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:19.993134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:20.018920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c55ba1-af5d-4a52-ba77-c285f2674822 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:20.025150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1432f8-db64-4e03-9ca7-b36fa435bc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:20.026670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1432f8-db64-4e03-9ca7-b36fa435bc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.027797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1432f8-db64-4e03-9ca7-b36fa435bc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.028917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1432f8-db64-4e03-9ca7-b36fa435bc2e admin admin] PATH_INFO: `/v3/projects/b4a17a8a0eab432b9c3820a7a19f6e23/users/e8c12c57c0704e7f821f34581c17612f/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.031791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1432f8-db64-4e03-9ca7-b36fa435bc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a17a8a0eab432b9c3820a7a19f6e23, user_id=e8c12c57c0704e7f821f34581c17612f, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:20.038268 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1432f8-db64-4e03-9ca7-b36fa435bc2e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:20.078597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 869/1772] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:19 2022] PUT /identity/v3/projects/b4a17a8a0eab432b9c3820a7a19f6e23/users/e8c12c57c0704e7f821f34581c17612f/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:20.093310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.093827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.094350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.106471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:20.117949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.121171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.121643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.125381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:20.128006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:20.159727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] MFA Rules not processed for user `e8c12c57c0704e7f821f34581c17612f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:20.161179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:20.195103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 904/1773] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:19 2022] POST /identity/v3/users => generated 283 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:20.198110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:20.228764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a7955-1ec3-426b-9581-b956bed98847 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:20.399020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 870/1774] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:20.413096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:20.414492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:20.414989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:20.472089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:20.473191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.475351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.475926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.477358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:20.487096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:20.533734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 905/1775] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:20.538853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:20.555247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa95b8-f283-4081-8ead-6c647cd57985 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:20.569051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:20.569806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.570069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.570318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.572256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12addcda-5897-423c-bae7-c3ca3c3b0591 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:20.582982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:20.595690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:20.596296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:20.597761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:20.599683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.600213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.600712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.602186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:20.613020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:20.639846 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 871/1776] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:20.655552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 906/1777] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:20.659712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:20.674870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e917-42d2-4124-b059-440a1b7098c6 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:20.707653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba06ac6-56c0-4bfc-b4fb-ebcc5c9f9440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:20.708896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba06ac6-56c0-4bfc-b4fb-ebcc5c9f9440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.709259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba06ac6-56c0-4bfc-b4fb-ebcc5c9f9440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.709573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba06ac6-56c0-4bfc-b4fb-ebcc5c9f9440 admin admin] PATH_INFO: `/v3/projects/733cb2cd320148edb48d834146a4de80/users/96522c5486aa428493e17f294bf52db0/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.750212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:20.755861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba06ac6-56c0-4bfc-b4fb-ebcc5c9f9440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733cb2cd320148edb48d834146a4de80, user_id=96522c5486aa428493e17f294bf52db0, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:20.757222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:20.774057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba06ac6-56c0-4bfc-b4fb-ebcc5c9f9440 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:20.832908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc0a0-7db7-4d7f-8dfd-c8d9bd99212e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:20.847183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 872/1778] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:20 2022] PUT /identity/v3/projects/733cb2cd320148edb48d834146a4de80/users/96522c5486aa428493e17f294bf52db0/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:20.865207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:20.867177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:20.868976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:20.960261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] MFA Rules not processed for user `96522c5486aa428493e17f294bf52db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:20.965047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:21.047022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:21.048479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:21.049084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:21.049688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] PATH_INFO: `/v3/projects/1ccb9efac24b421287425063cc965e93` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:21.052056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccb9efac24b421287425063cc965e93)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:21.054770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:21.139943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccb9efac24b421287425063cc965e93', 'request_id': 'req-b2220ddc-ec36-441c-ac7b-3a3f18decf40'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.153144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccb9efac24b421287425063cc965e93', 'request_id': 'req-b2220ddc-ec36-441c-ac7b-3a3f18decf40'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.186474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccb9efac24b421287425063cc965e93', 'request_id': 'req-b2220ddc-ec36-441c-ac7b-3a3f18decf40'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.224298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] The token cache is being invalidate because project 1ccb9efac24b421287425063cc965e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:21.224508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.228175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 873/1779] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:21.231398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:21.239483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.249808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 907/1780] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:20 2022] DELETE /identity/v3/projects/1ccb9efac24b421287425063cc965e93 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:21.250813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:21.267592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:21.273820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad5295-95be-4836-b116-f1a7fdb085db None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:21.491475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220ddc-ec36-441c-ac7b-3a3f18decf40 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:21.520738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:21.521623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:21.521909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:21.522209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:21.522340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:21.525723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:21.526293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:21.526798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] PATH_INFO: `/v3/users/f2ee5a475b1648699c31bf76a07ee62b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:21.528661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:21.531451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ee5a475b1648699c31bf76a07ee62b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:21.537410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:21.555448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:21.556939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:21.562249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:21.578280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 908/1781] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:21.579282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:21.608773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88c4a8-2c8e-46f7-9938-24a5b5848516 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:21.614250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:21.615296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:21.616007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:21.616354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:21.661384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:21.664198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:21.682858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 909/1782] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:21.686556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ee5a475b1648699c31bf76a07ee62b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.688446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:21.732989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ee5a475b1648699c31bf76a07ee62b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.734674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ee5a475b1648699c31bf76a07ee62b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.738176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231802e0-65f9-48f5-9bea-c7093e1576a5 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:21.763271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:21.767461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:21.771843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:21.772299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:21.775385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:21.775973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ee5a475b1648699c31bf76a07ee62b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.786929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:21.861188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ee5a475b1648699c31bf76a07ee62b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:21.873287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:21.881271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae876c1-b31b-4917-9904-3eb41f14969b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:21.884952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 874/1783] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:21 2022] DELETE /identity/v3/users/f2ee5a475b1648699c31bf76a07ee62b => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:21.886235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:21.891107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:21.928074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 910/1784] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:21.944973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:21.944973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:21.944973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.013203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:22.018930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:22.065135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b491-aa98-4a8c-ae5b-d783aa1e7db6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:22.071871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b201351-dace-441e-a4ff-def3041f6d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:22.073038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b201351-dace-441e-a4ff-def3041f6d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.073516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b201351-dace-441e-a4ff-def3041f6d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.074758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b201351-dace-441e-a4ff-def3041f6d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.075823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b201351-dace-441e-a4ff-def3041f6d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:22.079632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b201351-dace-441e-a4ff-def3041f6d71 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:22.140395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 875/1785] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:21 2022] POST /identity/v3/projects => generated 365 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:22.142935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c42c5-21d0-4e66-b817-d5b59a83cd02 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.147536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c42c5-21d0-4e66-b817-d5b59a83cd02 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.147992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c42c5-21d0-4e66-b817-d5b59a83cd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.194022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 911/1786] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:22.194753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:22.220047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03acfd2-8101-4f9f-b14e-b73e778aa078 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:22.226213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-076c42c5-21d0-4e66-b817-d5b59a83cd02 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:22.237363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-076c42c5-21d0-4e66-b817-d5b59a83cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:22.377329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:22.378339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.378811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.379343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.380233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:22.382467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:22.425790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 876/1787] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:22.426358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 912/1788] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:22 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:22.440396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:22.455325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.455622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.455911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.468136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3377e-f71c-49f3-8400-56897449fa1d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:22.489852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:22.491835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.491835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.494071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.496791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:22.504732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:22.532380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:22.535448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:22.544378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 913/1789] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:22.544943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:22.549161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:22.574072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 877/1790] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:22.576055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:22.591243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea15480-8983-4ed3-a8f9-a3cb967d6268 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:22.599403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:22.600721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.601017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.601301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] PATH_INFO: `/v3/projects/9f4c9d42aca94f7eb47c972f6b3394e2/users/f6ce30b066c74472b7352fb9df03dbf0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.651205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4c9d42aca94f7eb47c972f6b3394e2, user_id=f6ce30b066c74472b7352fb9df03dbf0, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:22.654528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:22.692984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 878/1791] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:22 2022] PUT /identity/v3/projects/9f4c9d42aca94f7eb47c972f6b3394e2/users/f6ce30b066c74472b7352fb9df03dbf0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:22.694036 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:22.762959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041dd88-7cc8-47c3-b12e-d45026deb287 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:22.773525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:22.840782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:22.946979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:22.948118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.948275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.948691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.949622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34f59c-106f-459f-93b0-5256af953642 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:22.963354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:22.967099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:22.967611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:22.968163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:22.969430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:22.969921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:22.971832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:22.990995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:22.994468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:22.995265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 879/1792] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:22.997843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.029104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d503697-71cf-4a8a-b14b-0351d27a8442 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.036065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 914/1793] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:23.038303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.061406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.064191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.064466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.064752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.070569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05097-2498-4f66-9559-7718ee80d8cd service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.075619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.082111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:23.085451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.086813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.087792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.088467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.090500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.094555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:23.130767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 880/1794] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:23.133199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.162094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7de493-ae23-41ed-be5e-bf210e46a984 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.173792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 915/1795] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:23 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:23.179365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.181990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622225c-e6df-40ff-a9d3-cbd8c669b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.185029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622225c-e6df-40ff-a9d3-cbd8c669b48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.185335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622225c-e6df-40ff-a9d3-cbd8c669b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.185621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622225c-e6df-40ff-a9d3-cbd8c669b48c admin admin] PATH_INFO: `/v3/projects/9f4c9d42aca94f7eb47c972f6b3394e2/users/f6ce30b066c74472b7352fb9df03dbf0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.188147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622225c-e6df-40ff-a9d3-cbd8c669b48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4c9d42aca94f7eb47c972f6b3394e2, user_id=f6ce30b066c74472b7352fb9df03dbf0, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.193193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622225c-e6df-40ff-a9d3-cbd8c669b48c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:23.212794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 881/1796] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:23 2022] PUT /identity/v3/projects/9f4c9d42aca94f7eb47c972f6b3394e2/users/f6ce30b066c74472b7352fb9df03dbf0/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:23.226235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.226235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.226235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.301941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] MFA Rules not processed for user `f6ce30b066c74472b7352fb9df03dbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:23.306749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:23.398273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec656b7-b4e3-4608-a16a-dac54fe40a33 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.528143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 882/1797] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:23.531000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.533672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:23.670994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.672026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.672898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.672898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.673571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.677249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:23.745316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8990c9-b030-4c6e-bb28-31de49cb4e32 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.760986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.764011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.764011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.764327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.769862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 916/1798] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:23 2022] POST /identity/v3/users => generated 294 bytes in 590 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:23.772990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.812139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad3c19-41c9-4190-b64e-707b01ab246c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.812721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.818804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:23.836840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.837562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.837956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.838208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.838976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.842277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:23.870021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 883/1799] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:23.891745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 917/1800] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:23.895175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:23.949591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9596-cd7b-4668-9d3d-b74f4f6e2df7 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:23.980526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:23.981320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:23.981622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:23.981908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:23.984006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:23.992205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:24.051151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:24.051364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 884/1801] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:24.067120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:24.094877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406d3da-44b3-4288-984e-060bc35b4f9f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:24.100762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38641e9-c6db-4379-8faf-76813120fd34 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:24.116511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:24.117860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22e9a4-95d9-4d83-bd2b-c18d06c8951e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:24.119954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:24.122114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e9a4-95d9-4d83-bd2b-c18d06c8951e admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:24.122600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e9a4-95d9-4d83-bd2b-c18d06c8951e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:24.123811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e9a4-95d9-4d83-bd2b-c18d06c8951e admin admin] PATH_INFO: `/v3/projects/c35838c41b04426dbb334256d54de176/users/a5a2fade07ea49108977923ac6456ce5/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:24.124002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:24.126161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:24.160849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:24.186342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22e9a4-95d9-4d83-bd2b-c18d06c8951e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35838c41b04426dbb334256d54de176, user_id=a5a2fade07ea49108977923ac6456ce5, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:24.189266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22e9a4-95d9-4d83-bd2b-c18d06c8951e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:24.204682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:24.231724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:24.245522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 885/1802] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:24 2022] PUT /identity/v3/projects/c35838c41b04426dbb334256d54de176/users/a5a2fade07ea49108977923ac6456ce5/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:24.275648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:24.276128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:24.276229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:24.353229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] MFA Rules not processed for user `a5a2fade07ea49108977923ac6456ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:24.354518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:24.520715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:24.522936 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:24.543545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 918/1803] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:24.644286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 886/1804] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:24.645539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:24.664522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7573f-4c7c-4a70-977f-9db505556a99 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:25.014872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27fa97-ffa0-4067-ae9a-bd055de60fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:25.015662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27fa97-ffa0-4067-ae9a-bd055de60fca admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:25.015955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27fa97-ffa0-4067-ae9a-bd055de60fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:25.023171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27fa97-ffa0-4067-ae9a-bd055de60fca admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:25.025697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27fa97-ffa0-4067-ae9a-bd055de60fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:25.045734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27fa97-ffa0-4067-ae9a-bd055de60fca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:25.173799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 919/1805] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:24 2022] POST /identity/v3/projects => generated 387 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:25.175168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:25.228450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92313181-a1a7-4c80-a29f-2854638cafa1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:25.380371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dfa15-d7aa-4183-a11e-a149fbc5b3d4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:25.380965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dfa15-d7aa-4183-a11e-a149fbc5b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:25.381420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dfa15-d7aa-4183-a11e-a149fbc5b3d4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:25.383628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 920/1806] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:25.414238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:25.414605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:25.415614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:25.500012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:25.501249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:25.596823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:25.597522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:25.597522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:25.597522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:25.598228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:25.600243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:25.689585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 887/1807] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:25 2022] POST /identity/v3/users => generated 300 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:25.690625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:25.734465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53b63c-65fc-42e3-92fa-b2b1ee5f2b24 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:25.742376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:25.743169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:25.743492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:25.743741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:25.744495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:25.747228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:25.783011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 888/1808] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:25.783996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:25.803678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939dcf0-c5b5-4b41-b2aa-fec7df763d2f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:25.810702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:25.812294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:25.813765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:25.814651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] PATH_INFO: `/v3/projects/467b5e0376cd42ccb0420ceeccbcc6a4/users/018285e5d3e84018baf92f0ef09cb091/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:25.856171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 921/1809] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:33:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:25.873781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] RBAC: Authorizing `identity:create_grant(project_id=467b5e0376cd42ccb0420ceeccbcc6a4, user_id=018285e5d3e84018baf92f0ef09cb091, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:25.876822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:25.927533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 889/1810] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:25 2022] PUT /identity/v3/projects/467b5e0376cd42ccb0420ceeccbcc6a4/users/018285e5d3e84018baf92f0ef09cb091/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:25.931731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:25.966917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3fc1-0117-4524-be34-fae5280d13c5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:26.078228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:26.195098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:26.197412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:26.198365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:26.201015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:26.204226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:26.207104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:26.237356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:26.240564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:26.265082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 922/1811] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:26.273797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:26.281997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c1303-dac5-44a6-a35a-1bb36ae2186f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:26.298545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:26.299510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:26.299801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:26.300146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] PATH_INFO: `/v3/projects/902f1fd8bcdc49849e8ce778bf1bc4ee` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:26.302725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=902f1fd8bcdc49849e8ce778bf1bc4ee)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:26.309897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:26.310907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364db2ea-f788-457d-bce3-1ba4f0ca8efd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:26.329419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3ba55-8690-43a5-a368-48a1d95bbedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:26.330366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ba55-8690-43a5-a368-48a1d95bbedf admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:26.330657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ba55-8690-43a5-a368-48a1d95bbedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:26.331123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ba55-8690-43a5-a368-48a1d95bbedf admin admin] PATH_INFO: `/v3/projects/467b5e0376cd42ccb0420ceeccbcc6a4/users/018285e5d3e84018baf92f0ef09cb091/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:26.341176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3ba55-8690-43a5-a368-48a1d95bbedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=467b5e0376cd42ccb0420ceeccbcc6a4, user_id=018285e5d3e84018baf92f0ef09cb091, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:26.349286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3ba55-8690-43a5-a368-48a1d95bbedf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:26.393066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 923/1812] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:26 2022] PUT /identity/v3/projects/467b5e0376cd42ccb0420ceeccbcc6a4/users/018285e5d3e84018baf92f0ef09cb091/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:26.398408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:26.398743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:26.399327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:26.427040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:26.494156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902f1fd8bcdc49849e8ce778bf1bc4ee', 'request_id': 'req-f3b34f53-1152-4f69-8e10-7ccad5188d06'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:26.516202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902f1fd8bcdc49849e8ce778bf1bc4ee', 'request_id': 'req-f3b34f53-1152-4f69-8e10-7ccad5188d06'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:26.544266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902f1fd8bcdc49849e8ce778bf1bc4ee', 'request_id': 'req-f3b34f53-1152-4f69-8e10-7ccad5188d06'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:26.554217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] MFA Rules not processed for user `018285e5d3e84018baf92f0ef09cb091`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:26.563381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:26.614221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] The token cache is being invalidate because project 902f1fd8bcdc49849e8ce778bf1bc4ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:26.615017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:26.615324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:26.620055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 890/1813] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:26 2022] DELETE /identity/v3/projects/902f1fd8bcdc49849e8ce778bf1bc4ee => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:26.621184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:26.629236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:26.862722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 924/1814] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:26.864104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:26.869044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:26.957164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34f53-1152-4f69-8e10-7ccad5188d06 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:26.979425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:26.980784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:26.980784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:26.980784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:26.990390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:27.029557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:27.031230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:27.048238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b1a33-ff6e-4d4f-bcc9-f180f31c692e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:27.051289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 891/1815] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:27.052161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:27.054065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:27.060654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:27.061844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:27.062346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:27.063037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] PATH_INFO: `/v3/users/e8c12c57c0704e7f821f34581c17612f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:27.064788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c12c57c0704e7f821f34581c17612f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:27.067386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:27.203587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:27.212966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42522ab8-638e-4c27-9ba0-fc075950c9c3 service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:27.233996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:27.234653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c12c57c0704e7f821f34581c17612f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:27.234776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:27.235860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:27.236159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:27.262880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:27.265442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:27.270881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c12c57c0704e7f821f34581c17612f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:27.296420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 892/1816] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:27.318375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c12c57c0704e7f821f34581c17612f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:27.320215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c12c57c0704e7f821f34581c17612f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:27.392858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b73ddbcf-5516-4cfa-80ff-438eb5fb209e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c12c57c0704e7f821f34581c17612f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:27.420190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 925/1817] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:26 2022] DELETE /identity/v3/users/e8c12c57c0704e7f821f34581c17612f => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:27.706001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:27.708678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:27.738232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e98151-d40d-4e70-9499-9287b51e3dcd service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:27.904776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:27.931623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:27.933346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:27.933916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:27.934873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] PATH_INFO: `/v3/users/a5a2fade07ea49108977923ac6456ce5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:27.937417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a2fade07ea49108977923ac6456ce5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:27.942382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:28.075682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a2fade07ea49108977923ac6456ce5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:28.119897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a2fade07ea49108977923ac6456ce5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:28.121252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a2fade07ea49108977923ac6456ce5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:28.139973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a2fade07ea49108977923ac6456ce5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:28.175695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a2fade07ea49108977923ac6456ce5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:28.191250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 893/1818] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:27 2022] DELETE /identity/v3/users/a5a2fade07ea49108977923ac6456ce5 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:28.197239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:28.197824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:28.198363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:28.255981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:28.258183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:28.405662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 926/1819] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:28.444308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:28.445847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:28.547750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073f38-ca6d-4566-940a-dfddc534d377 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:28.551835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63a199-52d1-48b6-bf51-3ab538ece3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:28.552955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63a199-52d1-48b6-bf51-3ab538ece3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:28.552955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63a199-52d1-48b6-bf51-3ab538ece3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:28.553174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63a199-52d1-48b6-bf51-3ab538ece3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:28.565397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63a199-52d1-48b6-bf51-3ab538ece3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:28.566939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63a199-52d1-48b6-bf51-3ab538ece3c6 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:28.578425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 894/1820] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:28.964103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:28.965824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:28.992675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889face-8548-4891-9adf-038cbd7f2586 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:28.999222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:29.000875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.001409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.001908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] PATH_INFO: `/v3/projects/b4a17a8a0eab432b9c3820a7a19f6e23` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.005373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a17a8a0eab432b9c3820a7a19f6e23)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:29.009794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:29.201102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a17a8a0eab432b9c3820a7a19f6e23', 'request_id': 'req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.231207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a17a8a0eab432b9c3820a7a19f6e23', 'request_id': 'req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.243610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a17a8a0eab432b9c3820a7a19f6e23', 'request_id': 'req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.273545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] The token cache is being invalidate because project b4a17a8a0eab432b9c3820a7a19f6e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:29.274207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.294972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a477d7d8-c2ca-48a0-83af-c6568f1b30ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.303319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 927/1821] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:28 2022] DELETE /identity/v3/projects/b4a17a8a0eab432b9c3820a7a19f6e23 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:29.310583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.311520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.311520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.369913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:29.371610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:29.522098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 895/1822] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:29.528369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.528685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.529745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.570775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:29.590828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:29.615156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 928/1823] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:29.616510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:29.633349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d03b5-1573-48ec-868f-6e02416b6f94 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:29.646498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:29.648240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.648716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.649298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.650328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:29.650805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:29.653736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:29.664209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:29.672303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 896/1824] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:29.674653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:29.680022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:29.681945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:29.687187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdb83d-c752-469b-8023-d94ab7301edf None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:29.693501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:29.694801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.695394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.695963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.696972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:29.700130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:29.702964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303f49-ea63-431b-a026-964490db3eac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:29.708311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:29.709512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.709968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.711218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] PATH_INFO: `/v3/users/96522c5486aa428493e17f294bf52db0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.712976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96522c5486aa428493e17f294bf52db0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:29.718429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:29.744472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 929/1825] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:29 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:29.748727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:29.769644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:29.805764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96522c5486aa428493e17f294bf52db0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.820746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96522c5486aa428493e17f294bf52db0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.821734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96522c5486aa428493e17f294bf52db0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.834636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96522c5486aa428493e17f294bf52db0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.852707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cfb2d2b8-e057-4f5d-bbb4-4cbfab8dced7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96522c5486aa428493e17f294bf52db0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:29.858139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c8c7c-3128-4f63-98c8-c64450a00413 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:29.861421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 897/1826] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:29 2022] DELETE /identity/v3/users/96522c5486aa428493e17f294bf52db0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:29.867277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.867804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.868230 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.869382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:29.871665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:29.872073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:29.873156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:29.874065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:29.876717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:29.895356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:29.897861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:29.910695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 930/1827] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:29 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:29.914328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:29.916637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:29.939117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b762f-1013-48b5-ba60-f5a6c36802aa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:29.986527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 898/1828] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:30.033419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.036157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:30.076876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.077666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.077954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.078235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.078986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.082906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.112369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 931/1829] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:30.113718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.116252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:30.149720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa50247-9196-43a1-a070-7250ad62b13b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.157758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e22fb-036f-4098-914c-87a7ee2729d6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.161381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.162428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.162888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.163431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] PATH_INFO: `/v3/projects/c35838c41b04426dbb334256d54de176` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.164999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35838c41b04426dbb334256d54de176)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.171026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.174347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.174910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.175224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.175505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.195167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.197155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.226904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 899/1830] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:30.228829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.241314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf6bb2-3146-4006-9538-d6af1b3d7263 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.264389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:30.272301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35838c41b04426dbb334256d54de176', 'request_id': 'req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.285770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35838c41b04426dbb334256d54de176', 'request_id': 'req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.296238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35838c41b04426dbb334256d54de176', 'request_id': 'req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.321488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] The token cache is being invalidate because project c35838c41b04426dbb334256d54de176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:30.321966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.322762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.330703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 932/1831] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:30 2022] DELETE /identity/v3/projects/c35838c41b04426dbb334256d54de176 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:30.331952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.333838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:30.358382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.359724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.359724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.359724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.361005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.362663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.372683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:30.379369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 900/1832] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:30.384301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.389305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:30.408963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05e6fd-b945-4255-8628-eee28d1c304b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.418762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21724-ecc6-470d-b3f5-0c4963f6fe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.419712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21724-ecc6-470d-b3f5-0c4963f6fe29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.420019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21724-ecc6-470d-b3f5-0c4963f6fe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.420300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21724-ecc6-470d-b3f5-0c4963f6fe29 admin admin] PATH_INFO: `/v3/projects/8c5ef06dda8d487a9fc109184bb62f5b/users/75385233f0cd49cda7405a4ad63598de/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.442826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c21724-ecc6-470d-b3f5-0c4963f6fe29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5ef06dda8d487a9fc109184bb62f5b, user_id=75385233f0cd49cda7405a4ad63598de, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.447434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c21724-ecc6-470d-b3f5-0c4963f6fe29 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.481195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1b0eb-9e70-41bb-b594-e56a9dac81e8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.483350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 901/1833] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:30 2022] PUT /identity/v3/projects/8c5ef06dda8d487a9fc109184bb62f5b/users/75385233f0cd49cda7405a4ad63598de/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:30.494731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.495141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.495566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.531264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] MFA Rules not processed for user `75385233f0cd49cda7405a4ad63598de`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:30.533042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:30.576132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.577273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.577701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.578145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] PATH_INFO: `/v3/users/018285e5d3e84018baf92f0ef09cb091` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.579734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] RBAC: Authorizing `identity:delete_user(user_id=018285e5d3e84018baf92f0ef09cb091)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.581881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.650304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 902/1834] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:30.655542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.674227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0c4e2-f31a-46a1-a637-2b9239bd33e9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.679244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.680028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.680505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.680787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.680980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018285e5d3e84018baf92f0ef09cb091'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.682084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.685280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.713426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018285e5d3e84018baf92f0ef09cb091'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.732840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 903/1835] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:30 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:30.736413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.741599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018285e5d3e84018baf92f0ef09cb091'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.742992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018285e5d3e84018baf92f0ef09cb091'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.755221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018285e5d3e84018baf92f0ef09cb091'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:30.765585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 933/1836] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:30 2022] DELETE /identity/v3/users/018285e5d3e84018baf92f0ef09cb091 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:30.817857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:30.826993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cadf-c628-4ae9-b7eb-a07f0854b9a8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.910037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4fece-b112-42b1-a478-4e44ea846440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.910934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4fece-b112-42b1-a478-4e44ea846440 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.911249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4fece-b112-42b1-a478-4e44ea846440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.911587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4fece-b112-42b1-a478-4e44ea846440 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.912994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4fece-b112-42b1-a478-4e44ea846440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.916323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4fece-b112-42b1-a478-4e44ea846440 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.944246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 904/1837] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:30 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:30.948948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:30.950634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:30.967076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad33f0-2705-489a-9fca-885e180ca97f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:30.972011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3fa9f-a588-43e0-9fa9-fda24709f15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:30.972696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3fa9f-a588-43e0-9fa9-fda24709f15f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.972980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3fa9f-a588-43e0-9fa9-fda24709f15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.973483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3fa9f-a588-43e0-9fa9-fda24709f15f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.973991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3fa9f-a588-43e0-9fa9-fda24709f15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:30.976396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3fa9f-a588-43e0-9fa9-fda24709f15f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:30.990316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c44ef-ce8a-485e-a51a-0a7c25513b14 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:30.990604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c44ef-ce8a-485e-a51a-0a7c25513b14 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:30.990884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c44ef-ce8a-485e-a51a-0a7c25513b14 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:30.993093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 905/1838] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:33:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:30.994110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 934/1839] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:30.999766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c44ef-ce8a-485e-a51a-0a7c25513b14 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.010294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.016040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.016040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.019907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c44ef-ce8a-485e-a51a-0a7c25513b14 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.035805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.036852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.037139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.037417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] PATH_INFO: `/v3/projects/c84147fd5d36454eb900838c2269c20c/users/fc7d78845aa04bd5abab34cffbd610d1/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.047226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:31.048567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.080075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84147fd5d36454eb900838c2269c20c, user_id=fc7d78845aa04bd5abab34cffbd610d1, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.083134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.124049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:31.147229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 906/1840] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:30 2022] PUT /identity/v3/projects/c84147fd5d36454eb900838c2269c20c/users/fc7d78845aa04bd5abab34cffbd610d1/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:31.153150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.245739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 935/1841] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:33:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:31.247565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.250493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.272061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7103a-f8a2-4971-a979-92467b6bc927 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.274474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9acc2b-7294-4b2d-9cbf-41679025e5e1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.281931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.283400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.284368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.285015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.287716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.288619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.291414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.292150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.292698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] PATH_INFO: `/v3/projects/733cb2cd320148edb48d834146a4de80` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.294037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.295685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=733cb2cd320148edb48d834146a4de80)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.299582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.332037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 907/1842] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:31.341108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.348052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.364518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69639c-2f49-4483-8a2d-314f9edfff4a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.380016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.381764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.382683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.383219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] PATH_INFO: `/v3/projects/467b5e0376cd42ccb0420ceeccbcc6a4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.385982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=467b5e0376cd42ccb0420ceeccbcc6a4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.390567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.399842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733cb2cd320148edb48d834146a4de80', 'request_id': 'req-f11c98ef-7677-4e25-aba0-0eac162a06d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.408170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733cb2cd320148edb48d834146a4de80', 'request_id': 'req-f11c98ef-7677-4e25-aba0-0eac162a06d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.436362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733cb2cd320148edb48d834146a4de80', 'request_id': 'req-f11c98ef-7677-4e25-aba0-0eac162a06d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.457829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] The token cache is being invalidate because project 733cb2cd320148edb48d834146a4de80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:31.458123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.458827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.460931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 936/1843] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:31 2022] DELETE /identity/v3/projects/733cb2cd320148edb48d834146a4de80 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:31.461902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.463350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.482710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467b5e0376cd42ccb0420ceeccbcc6a4', 'request_id': 'req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.491518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467b5e0376cd42ccb0420ceeccbcc6a4', 'request_id': 'req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.493647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:31.500985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467b5e0376cd42ccb0420ceeccbcc6a4', 'request_id': 'req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.513285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] The token cache is being invalidate because project 467b5e0376cd42ccb0420ceeccbcc6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:31.513568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.514240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.515929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 908/1844] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:31 2022] DELETE /identity/v3/projects/467b5e0376cd42ccb0420ceeccbcc6a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:31.516693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.517978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.536218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c98ef-7677-4e25-aba0-0eac162a06d0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.588204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:31.595912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c95a9-9325-4ef5-b6d9-5c49e32e5d0c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.596094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.596727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.597188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.597473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] PATH_INFO: `/v3/projects/c84147fd5d36454eb900838c2269c20c/users/fc7d78845aa04bd5abab34cffbd610d1/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.600097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84147fd5d36454eb900838c2269c20c, user_id=fc7d78845aa04bd5abab34cffbd610d1, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.601686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.602344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.602631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.602912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.602999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.605536 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.615172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 937/1845] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:31 2022] PUT /identity/v3/projects/c84147fd5d36454eb900838c2269c20c/users/fc7d78845aa04bd5abab34cffbd610d1/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:31.616310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.617896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.627903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.629559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5cea6-0575-4e91-ad0c-0dd6a676c17e service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.641697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 909/1846] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:31.643724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a28c10-1a75-4d3f-a94b-fa4885cf598e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.643724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a28c10-1a75-4d3f-a94b-fa4885cf598e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.643724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a28c10-1a75-4d3f-a94b-fa4885cf598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.666397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-39a28c10-1a75-4d3f-a94b-fa4885cf598e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:31.668812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-39a28c10-1a75-4d3f-a94b-fa4885cf598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.705510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb054384-45c2-4d22-b47e-b68eff4261fc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.711792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:31.712692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.715738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.715738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] PATH_INFO: `/v3/users/f6ce30b066c74472b7352fb9df03dbf0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.716259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ce30b066c74472b7352fb9df03dbf0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:31.718331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:31.769436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 910/1847] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:31.771452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.771452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.771752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.798988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ce30b066c74472b7352fb9df03dbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.802277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] MFA Rules not processed for user `fc7d78845aa04bd5abab34cffbd610d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:31.803793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.808993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ce30b066c74472b7352fb9df03dbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.824321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ce30b066c74472b7352fb9df03dbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.826000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ce30b066c74472b7352fb9df03dbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.834934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48e550ea-c3e8-4b12-a07b-af050c34dced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ce30b066c74472b7352fb9df03dbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:31.842417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 938/1848] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:31 2022] DELETE /identity/v3/users/f6ce30b066c74472b7352fb9df03dbf0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:31.846893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:31.846893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:31.846893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:31.873732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:31.874966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.886440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:31.903725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 911/1849] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:31.918456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.920305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:31.981357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 939/1850] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:31.985882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:31.997104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21874efb-4a82-4c4b-ae28-e33c7385d19a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:31.997411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640026f0-6c3a-4fb3-85fe-782d040772e7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.002725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.004439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.004710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.005009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.005723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:32.006027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.007849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.016447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:32.027309 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 940/1851] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:33:32.029366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.041010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf675-3df9-4161-b0bd-835916d2fa19 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.050043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.050788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.051139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.051393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.052250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.054450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.070000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.070668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.070948 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.071253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.105666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 941/1852] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:32 2022] POST /identity/v3/projects => generated 359 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:32.106643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.171559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.173030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.194140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 912/1853] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:32.226135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.236219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:32.261718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fb014-4bb5-437b-8b2a-c2ab55d566f3 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.262338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c448ee-9463-4031-98bc-ecfbde42a983 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.287645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.289786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.290555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.290872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.291124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.291810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.294645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.296737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.296897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.297153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] PATH_INFO: `/v3/projects/9f4c9d42aca94f7eb47c972f6b3394e2` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.300796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4c9d42aca94f7eb47c972f6b3394e2)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.300796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.341888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 942/1854] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:32 2022] POST /identity/v3/users => generated 280 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:32.346005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.370826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4c9d42aca94f7eb47c972f6b3394e2', 'request_id': 'req-08b45ef2-1929-4a69-8ae2-c26194ff688e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:32.380513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195750-a479-47e0-8629-4545c8f0a9a4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.381223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4c9d42aca94f7eb47c972f6b3394e2', 'request_id': 'req-08b45ef2-1929-4a69-8ae2-c26194ff688e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:32.387178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:32.394311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4c9d42aca94f7eb47c972f6b3394e2', 'request_id': 'req-08b45ef2-1929-4a69-8ae2-c26194ff688e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:32.413089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] The token cache is being invalidate because project 9f4c9d42aca94f7eb47c972f6b3394e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:32.418279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:32.419246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-08b45ef2-1929-4a69-8ae2-c26194ff688e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:32.423952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 913/1855] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:32 2022] DELETE /identity/v3/projects/9f4c9d42aca94f7eb47c972f6b3394e2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:32.430311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d20a1-b4c5-40c3-bc5a-9a031774bc5d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.430638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d20a1-b4c5-40c3-bc5a-9a031774bc5d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.430879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d20a1-b4c5-40c3-bc5a-9a031774bc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.457257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f33d20a1-b4c5-40c3-bc5a-9a031774bc5d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:32.458488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f33d20a1-b4c5-40c3-bc5a-9a031774bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:32.526698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.528677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.529341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.529829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.530786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.534424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.561282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 943/1856] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:32.565507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.568338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:32.590359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17b68-c096-4c8d-bb9d-48fa9caf4619 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.599527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.600775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.601327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.601603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.602585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.605136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.619926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 944/1857] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:32.624526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.641492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 914/1858] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:32.644288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb543c25-5cf1-45d8-8f17-ac04cccca4d9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.649599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.650111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.650601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.650982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8f150-9caa-4b81-a0c2-20acabf2ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.653009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f150-9caa-4b81-a0c2-20acabf2ffe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.653097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f150-9caa-4b81-a0c2-20acabf2ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.653336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f150-9caa-4b81-a0c2-20acabf2ffe0 admin admin] PATH_INFO: `/v3/projects/4020c4b6fdcc42b78596ad8b9f5dd19c/users/8abf99294ad14f06bc25b9ea574cb4f0/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.688598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8f150-9caa-4b81-a0c2-20acabf2ffe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4020c4b6fdcc42b78596ad8b9f5dd19c, user_id=8abf99294ad14f06bc25b9ea574cb4f0, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.691977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:32.693747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:32.695354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8f150-9caa-4b81-a0c2-20acabf2ffe0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.715235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 915/1859] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:32.719708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.727357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 945/1860] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:32 2022] PUT /identity/v3/projects/4020c4b6fdcc42b78596ad8b9f5dd19c/users/8abf99294ad14f06bc25b9ea574cb4f0/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:32.728914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.729207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.730092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.742255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91ee3d-60fd-4749-8b12-57f7535a0a16 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.775600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] MFA Rules not processed for user `8abf99294ad14f06bc25b9ea574cb4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:32.778181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:32.795521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:32.863985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.864669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.864951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.865228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.865960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:32.866265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.867929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:32.898964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 916/1861] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:33:32.903858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:32.910923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 946/1862] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:32 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:32.920648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c4f3a-c058-4c93-b15b-30f8a85b7d85 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:32.928754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:32.931107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:32.931980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:32.932423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:32.935503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:32.937507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.011386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.011951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 917/1863] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:32 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:33.039444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b6e26-b8b0-421d-8b59-059274fe8c8a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.052710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.054676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:33.173666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.173928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.174395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.174852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.182010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.185088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.220761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:33.226450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 947/1864] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:33 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:33.239104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.242367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:33.256685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d7ddd-6efa-4f84-872d-dbd8be6772cf admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.274576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.275672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.276053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.276345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] PATH_INFO: `/v3/users/75385233f0cd49cda7405a4ad63598de` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.280226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75385233f0cd49cda7405a4ad63598de)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.284010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.378484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c27ea0-2d59-49ae-ae2c-a3d117c726c7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.393130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a35ca5-b662-4ada-a4c0-ff3cc9c3c6cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.394017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35ca5-b662-4ada-a4c0-ff3cc9c3c6cb service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.394576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35ca5-b662-4ada-a4c0-ff3cc9c3c6cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.395119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35ca5-b662-4ada-a4c0-ff3cc9c3c6cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.423944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75385233f0cd49cda7405a4ad63598de'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.496436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75385233f0cd49cda7405a4ad63598de'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.498191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75385233f0cd49cda7405a4ad63598de'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.511278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75385233f0cd49cda7405a4ad63598de'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.535488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75385233f0cd49cda7405a4ad63598de'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.552723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 918/1865] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:33 2022] DELETE /identity/v3/users/75385233f0cd49cda7405a4ad63598de => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:33.556698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.562103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:33.576473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a35ca5-b662-4ada-a4c0-ff3cc9c3c6cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.579520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a35ca5-b662-4ada-a4c0-ff3cc9c3c6cb service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.590756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:33.605400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 948/1866] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:33.606949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.607278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.607567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.631166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:33.632516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:33.675402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46093-3668-4d5e-b075-e275d57b3d62 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.682376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.683667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.684478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.684768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.685530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.687651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.712462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 949/1867] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:33.714230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.714423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 919/1868] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:33.715816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:33.722783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.736779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daacfaa-f4f4-463b-b398-1e94a410b930 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.752608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b42d82-104c-4b68-bf3f-0d44a17619ac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.756123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.757597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.758458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.758634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] PATH_INFO: `/v3/users/8abf99294ad14f06bc25b9ea574cb4f0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.761665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abf99294ad14f06bc25b9ea574cb4f0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.765503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.766973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.767145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.767338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.767577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.768392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.771709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.787716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 920/1869] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:33.788480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.792704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:33.871704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abf99294ad14f06bc25b9ea574cb4f0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.893342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abf99294ad14f06bc25b9ea574cb4f0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.903329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3b32a-cb3a-48c1-b049-5f4363c05887 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.907753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:33.908685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.908807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.909376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.916427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abf99294ad14f06bc25b9ea574cb4f0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.917892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abf99294ad14f06bc25b9ea574cb4f0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.924094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:33.925592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:33.929042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eecc0c23-3208-42b6-82aa-a5dd88eea14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abf99294ad14f06bc25b9ea574cb4f0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:33.932048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:33.938561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 921/1870] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:33.939839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:33.947221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 950/1871] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:33 2022] DELETE /identity/v3/users/8abf99294ad14f06bc25b9ea574cb4f0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:33.952063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c711fc-c1b3-4e00-ba6f-bb07d1808fcf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:33.952685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c711fc-c1b3-4e00-ba6f-bb07d1808fcf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:33.953002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c711fc-c1b3-4e00-ba6f-bb07d1808fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:33.966418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d7fe5-aca0-4356-8843-c031dac429bc service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:33.998193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-99c711fc-c1b3-4e00-ba6f-bb07d1808fcf None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:34.002411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-99c711fc-c1b3-4e00-ba6f-bb07d1808fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:34.068091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:34.068504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.068895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.069683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] PATH_INFO: `/v3/projects/4119c41e307448af9a0e64e1365c6ec1/users/7290f802ccdc4b8199f075690c0bbef8/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.088815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4119c41e307448af9a0e64e1365c6ec1, user_id=7290f802ccdc4b8199f075690c0bbef8, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:34.092932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:34.108190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 951/1872] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:34.116547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 922/1873] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:33 2022] PUT /identity/v3/projects/4119c41e307448af9a0e64e1365c6ec1/users/7290f802ccdc4b8199f075690c0bbef8/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:34.127198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.127542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.127775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.170995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:34.177867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] MFA Rules not processed for user `7290f802ccdc4b8199f075690c0bbef8`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:34.180589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:34.222501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:34.225897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:34.345390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 952/1874] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:34.347195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:34.349556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:34.351099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038927f5-750b-4378-a894-ffa2b5c905ba admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:34.358658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:34.359895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.360373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.360802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.441274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b8a48-47d6-4514-8420-fc5a736731d5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:34.447263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:34.448477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.448988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.450426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.450426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:34.452457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:34.515516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 953/1875] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:34 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:34.516646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:34.517609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:34.519575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:34.548184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 923/1876] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:34.641982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126479d9-4504-47c4-9c71-a9960b313c2f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:34.649394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:34.650533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.651133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.651629 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.652724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:34.654769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:34.695466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:34.697540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 954/1877] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:34 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:34.716163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28fe8b-73aa-43fd-bf00-c2968c0e1dbd service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:34.730232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:34.731193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.731486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.731904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.732653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:34.734824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:34.767102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 924/1878] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:34.771756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:34.786209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf1a38-90de-403f-b3a7-945a2cb8d688 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:34.792301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:34.794267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:34.794797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:34.795336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] PATH_INFO: `/v3/projects/485fa9af6d4f46e9be07f479116fb260/users/edad1f617e6c47d08e606a5a0b42ca4b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:34.836784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485fa9af6d4f46e9be07f479116fb260, user_id=edad1f617e6c47d08e606a5a0b42ca4b, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:34.840076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:34.842782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:34.872005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 955/1879] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:34 2022] PUT /identity/v3/projects/485fa9af6d4f46e9be07f479116fb260/users/edad1f617e6c47d08e606a5a0b42ca4b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:34.872624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:34.892014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab79eb-4565-4213-9193-100faddc4d49 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:34.934109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:35.005661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917511e5-b965-4796-9fd6-d111ea3f453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:35.006508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917511e5-b965-4796-9fd6-d111ea3f453a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.006888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917511e5-b965-4796-9fd6-d111ea3f453a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.007232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917511e5-b965-4796-9fd6-d111ea3f453a admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.008086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917511e5-b965-4796-9fd6-d111ea3f453a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:35.009868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917511e5-b965-4796-9fd6-d111ea3f453a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:35.024509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 925/1880] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:35.029369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:35.042058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf7c9-f347-4dcd-b21a-d45132ac7b47 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:35.049135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:35.050130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.050493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.050671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] PATH_INFO: `/v3/projects/485fa9af6d4f46e9be07f479116fb260/users/edad1f617e6c47d08e606a5a0b42ca4b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.057185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] RBAC: Authorizing `identity:create_grant(project_id=485fa9af6d4f46e9be07f479116fb260, user_id=edad1f617e6c47d08e606a5a0b42ca4b, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:35.061566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:35.079552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 956/1881] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:35 2022] PUT /identity/v3/projects/485fa9af6d4f46e9be07f479116fb260/users/edad1f617e6c47d08e606a5a0b42ca4b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:35.086947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.087304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.087552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.159386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] MFA Rules not processed for user `edad1f617e6c47d08e606a5a0b42ca4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:35.160764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:35.365417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:35.367205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:35.374899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 926/1882] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:35.383538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:35.388115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:35.479149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c972439-33a5-4d63-b8df-736c54d8247b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:35.488244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:35.488244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.488244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.488244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] PATH_INFO: `/v3/users/ea60343fe4be4f7983b0b6591bf79015` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.488735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea60343fe4be4f7983b0b6591bf79015)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:35.489004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:35.515805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323f392-1b1a-4bb9-8d69-8753c619a4ec None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:35.519567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:35.520228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.520586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.520779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.541699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:35.543834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:35.579250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 927/1883] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:35.580362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:35.587583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:35.608663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea60343fe4be4f7983b0b6591bf79015'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.618349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023ab31-b89b-4438-bc26-34fc1fe25a38 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:35.638927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:35.638927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.638927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.638927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] PATH_INFO: `/v3/projects/8c5ef06dda8d487a9fc109184bb62f5b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.647998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5ef06dda8d487a9fc109184bb62f5b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:35.650126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:35.661502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea60343fe4be4f7983b0b6591bf79015'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.732995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea60343fe4be4f7983b0b6591bf79015'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.736949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea60343fe4be4f7983b0b6591bf79015'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.772972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2a8bef70-edf1-43b3-abd2-b5a59caf0108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea60343fe4be4f7983b0b6591bf79015'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.786133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:35.795465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 957/1884] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:35 2022] DELETE /identity/v3/users/ea60343fe4be4f7983b0b6591bf79015 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:35.807130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:35.808752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:35.809293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:35.833463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5ef06dda8d487a9fc109184bb62f5b', 'request_id': 'req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.862457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:35.865937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5ef06dda8d487a9fc109184bb62f5b', 'request_id': 'req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.869822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:35.884650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5ef06dda8d487a9fc109184bb62f5b', 'request_id': 'req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.980041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] The token cache is being invalidate because project 8c5ef06dda8d487a9fc109184bb62f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:35.980575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.988385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:35.994794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 928/1885] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:35 2022] DELETE /identity/v3/projects/8c5ef06dda8d487a9fc109184bb62f5b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:35.999402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:36.001575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:36.120593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:36.164237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 958/1886] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:36.204787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc6c38-6a31-4dd4-8e09-2c2e10c8ebd9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:36.216308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:36.217837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:36.218343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:36.218900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] PATH_INFO: `/v3/users/fc7d78845aa04bd5abab34cffbd610d1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:36.222777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7d78845aa04bd5abab34cffbd610d1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:36.223827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:36.226335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:36.226873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:36.380451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7d78845aa04bd5abab34cffbd610d1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:36.412722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7d78845aa04bd5abab34cffbd610d1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:36.413757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7d78845aa04bd5abab34cffbd610d1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:36.433256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77031af7-cc44-4962-868b-b1293903a3fb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:36.437711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:36.438388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:36.438675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:36.438966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:36.440970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7d78845aa04bd5abab34cffbd610d1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:36.441276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:36.458723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:36.461083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:36.474653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 959/1887] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:36.475618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:36.477909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:36.490835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7d78845aa04bd5abab34cffbd610d1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:36.504813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 929/1888] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:35 2022] DELETE /identity/v3/users/fc7d78845aa04bd5abab34cffbd610d1 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:36.624612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06960b7f-f1ab-47c2-a07f-9a99f2761c67 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:36.788114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edad1f617e6c47d08e606a5a0b42ca4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485fa9af6d4f46e9be07f479116fb260', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:36.789539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:36.790043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:36.791001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:36.795753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:36.796388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:36.801396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:36.831188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 960/1889] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:36.835448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:36.865553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf93a2-ceb0-4ccc-9195-1b3dd6691c53 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:36.877805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edad1f617e6c47d08e606a5a0b42ca4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485fa9af6d4f46e9be07f479116fb260', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:36.879528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:36.880366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:36.880897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:36.882939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:36.894315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:36.962373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 930/1890] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:36 2022] POST /identity/v3/projects => generated 353 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:36.967519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:37.005429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718f011-9a7e-4140-b23b-3d97c7c7a13f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:37.188462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:37.192037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:37.225387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:37.322119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edad1f617e6c47d08e606a5a0b42ca4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485fa9af6d4f46e9be07f479116fb260', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:37.322833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:37.323950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:37.324416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:37.332246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:37.332795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:37.334517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:37.372328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:37.404421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 961/1891] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:33:37.464871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90087d1-b920-443c-af2d-e943898e3d1f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:37.472061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:37.473641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:37.474150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:37.474862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] PATH_INFO: `/v3/projects/4020c4b6fdcc42b78596ad8b9f5dd19c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:37.477774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4020c4b6fdcc42b78596ad8b9f5dd19c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:37.483087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:37.496091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:37.519403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf192-f17e-4730-b5be-557e82afcefb tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:37.535480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edad1f617e6c47d08e606a5a0b42ca4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485fa9af6d4f46e9be07f479116fb260', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:37.536641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:37.537245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:37.537558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] PATH_INFO: `/v3/projects/953b0e8c2f1548269900c376e60a8c7c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:37.539851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorizing `identity:delete_project(project_id=953b0e8c2f1548269900c376e60a8c7c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:37.544523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:37.559233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4020c4b6fdcc42b78596ad8b9f5dd19c', 'request_id': 'req-4b688cd4-e205-42a1-8b2a-f5e678efb57d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.576145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4020c4b6fdcc42b78596ad8b9f5dd19c', 'request_id': 'req-4b688cd4-e205-42a1-8b2a-f5e678efb57d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.594053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4020c4b6fdcc42b78596ad8b9f5dd19c', 'request_id': 'req-4b688cd4-e205-42a1-8b2a-f5e678efb57d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.623169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] The token cache is being invalidate because project 4020c4b6fdcc42b78596ad8b9f5dd19c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:37.623735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.624847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b688cd4-e205-42a1-8b2a-f5e678efb57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.628585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 931/1892] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:37 2022] DELETE /identity/v3/projects/4020c4b6fdcc42b78596ad8b9f5dd19c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:37.635033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2d7d-e90a-4d71-b769-b404a6dcefe0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:37.637105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2d7d-e90a-4d71-b769-b404a6dcefe0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:37.637914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2d7d-e90a-4d71-b769-b404a6dcefe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:37.693300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-127b2d7d-e90a-4d71-b769-b404a6dcefe0 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:37.697121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-127b2d7d-e90a-4d71-b769-b404a6dcefe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:37.731793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953b0e8c2f1548269900c376e60a8c7c', 'request_id': 'req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.780559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953b0e8c2f1548269900c376e60a8c7c', 'request_id': 'req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.813831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953b0e8c2f1548269900c376e60a8c7c', 'request_id': 'req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.852900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] The token cache is being invalidate because project 953b0e8c2f1548269900c376e60a8c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:37.853722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.873002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:37.875880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 962/1893] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:37 2022] DELETE /identity/v3/projects/953b0e8c2f1548269900c376e60a8c7c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:37.910062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:37.912819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:37.950031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 932/1894] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:37.952281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:37.952597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:37.953570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:37.972591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95c415-e4a2-4985-8e2c-118c1f3cd50f tempest-VolumeQuotaClassesTest-404911539 tempest-VolumeQuotaClassesTest-404911539-project-admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:37.989360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:37.992431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:37.994160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:37.995910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:37.996543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:37.997104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] PATH_INFO: `/v3/projects/c84147fd5d36454eb900838c2269c20c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:37.999742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84147fd5d36454eb900838c2269c20c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:38.005553 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:38.030722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 933/1895] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:38.040758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:38.066106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344f168-115d-4212-9fcd-5cc9ec0b529a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:38.094431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:38.096441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:38.097310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:38.107180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:38.109879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:38.110426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:38.116346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:38.144813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84147fd5d36454eb900838c2269c20c', 'request_id': 'req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:38.162432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 934/1896] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:38.164271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84147fd5d36454eb900838c2269c20c', 'request_id': 'req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:38.164929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:38.192802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84147fd5d36454eb900838c2269c20c', 'request_id': 'req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:38.197903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86844bf-2427-44da-89e6-0906c17379f3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:38.237130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] The token cache is being invalidate because project c84147fd5d36454eb900838c2269c20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:38.237555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:38.242737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd3e2257-25a3-47b1-af5a-bf0bd74c5677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:38.258218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 963/1897] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:37 2022] DELETE /identity/v3/projects/c84147fd5d36454eb900838c2269c20c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:38.267852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:38.268718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:38.269493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:38.284864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:38.335394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:38.345952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:38.373044 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:38.373350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:38.374495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:38.380190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:38.385354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:38.402998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:38.499369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 935/1898] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:38 2022] POST /identity/v3/projects => generated 375 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:38.500065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:38.511739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:38.531569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:38.605772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 964/1899] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:38.607776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:38.610785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:38.630538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4fb3a-b3b7-4fc5-bf30-5b9dc2376a27 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:38.739733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793f25-38a5-4a27-9d0f-867863432898 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:38.780619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:38.786780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:38.787280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:38.787822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] PATH_INFO: `/v3/users/7290f802ccdc4b8199f075690c0bbef8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:38.791486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7290f802ccdc4b8199f075690c0bbef8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:38.801524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:38.835029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dff37a-6857-43b3-94ca-ca62a494c242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:38.837421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff37a-6857-43b3-94ca-ca62a494c242 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:38.838348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff37a-6857-43b3-94ca-ca62a494c242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:38.839199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff37a-6857-43b3-94ca-ca62a494c242 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:38.843039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dff37a-6857-43b3-94ca-ca62a494c242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:38.847747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dff37a-6857-43b3-94ca-ca62a494c242 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:38.966129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 965/1900] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:38 2022] POST /identity/v3/users => generated 288 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:38.971167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:38.971648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:38.972163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:38.990035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7290f802ccdc4b8199f075690c0bbef8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.019512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7290f802ccdc4b8199f075690c0bbef8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.020827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:39.021291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7290f802ccdc4b8199f075690c0bbef8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.023242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.058934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 966/1901] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:39.060638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.063305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.071441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7290f802ccdc4b8199f075690c0bbef8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.103803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de5e7-04b4-45c3-9a57-03dde819f54f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:39.124412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:39.124412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:39.124412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:39.127638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] PATH_INFO: `/v3/projects/75b642b9055c4317b3efbbfd1b3786dd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:39.130671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b642b9055c4317b3efbbfd1b3786dd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:39.135093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:39.147222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7290f802ccdc4b8199f075690c0bbef8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.170589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 936/1902] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:38 2022] DELETE /identity/v3/users/7290f802ccdc4b8199f075690c0bbef8 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:39.172275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.177033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.302480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b642b9055c4317b3efbbfd1b3786dd', 'request_id': 'req-8366571d-2916-46e2-bbcf-7fed9f5bba62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.324064 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b642b9055c4317b3efbbfd1b3786dd', 'request_id': 'req-8366571d-2916-46e2-bbcf-7fed9f5bba62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.338315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:39.342027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b642b9055c4317b3efbbfd1b3786dd', 'request_id': 'req-8366571d-2916-46e2-bbcf-7fed9f5bba62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.402766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] The token cache is being invalidate because project 75b642b9055c4317b3efbbfd1b3786dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:39.403421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.404732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.408792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 967/1903] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:39 2022] DELETE /identity/v3/projects/75b642b9055c4317b3efbbfd1b3786dd => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:39.412147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.428395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.434548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116973-a661-40a4-aa86-ee540f774ce4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:39.581082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd710474-609c-4ab6-adbb-1684562bcd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:39.582324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd710474-609c-4ab6-adbb-1684562bcd67 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:39.582687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd710474-609c-4ab6-adbb-1684562bcd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:39.583140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd710474-609c-4ab6-adbb-1684562bcd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:39.584486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd710474-609c-4ab6-adbb-1684562bcd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:39.587590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd710474-609c-4ab6-adbb-1684562bcd67 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:39.606830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 937/1904] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:39.610431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:39.611121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:39.612790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:39.632570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:39.669482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:39.670745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.682162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 938/1905] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:39.683476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.685823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.693111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366571d-2916-46e2-bbcf-7fed9f5bba62 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:39.699244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddb4f9-a681-4259-93aa-a040f044b3c4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:39.705792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:39.706883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:39.707232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:39.707578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] PATH_INFO: `/v3/users/ee1dec4ce04d43868d1c4318ec71ab62` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:39.708813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1dec4ce04d43868d1c4318ec71ab62)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:39.708992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:39.710241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:39.710744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:39.711027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:39.711282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:39.712940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:39.714079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:39.718394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:39.738606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 968/1906] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:39.739525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.741270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.756843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb593789-97e9-4026-9631-4a5b76328fa8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:39.771427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:39.772835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:39.773402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:39.773944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:39.775391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:39.779658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:39.792516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1dec4ce04d43868d1c4318ec71ab62'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.797350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 969/1907] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:39.798318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.800368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.815736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1dec4ce04d43868d1c4318ec71ab62'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.817072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1dec4ce04d43868d1c4318ec71ab62'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.834037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1dec4ce04d43868d1c4318ec71ab62'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.856959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1dec4ce04d43868d1c4318ec71ab62'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:39.872972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 939/1908] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:39 2022] DELETE /identity/v3/users/ee1dec4ce04d43868d1c4318ec71ab62 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:39.875140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:39.881487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:39.915316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868bba7-e360-49ef-a0c9-39f89f244459 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.006668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:40.012405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.013737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.014367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.014955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.018978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.037436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef45cce-68d8-44c6-b5cd-056329300234 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.046343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.047166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.047587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.048098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.048566 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.049920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.050372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.053626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.071654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 970/1909] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:40.074349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.076877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.099660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af08af7-db86-4a6a-bf03-00ca3347b078 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.105989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 940/1910] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:39 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:40.108058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.239720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5a6c1-3886-4fdc-98a7-2c75e22727d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.242174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5a6c1-3886-4fdc-98a7-2c75e22727d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.242566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5a6c1-3886-4fdc-98a7-2c75e22727d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.242854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5a6c1-3886-4fdc-98a7-2c75e22727d8 admin admin] PATH_INFO: `/v3/projects/5f080ee06c10413a83ab2add242d58b5/users/83a90b57c53348199830423d46cdf223/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.270687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5a6c1-3886-4fdc-98a7-2c75e22727d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f080ee06c10413a83ab2add242d58b5, user_id=83a90b57c53348199830423d46cdf223, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.275369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5a6c1-3886-4fdc-98a7-2c75e22727d8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.302172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 971/1911] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:40 2022] PUT /identity/v3/projects/5f080ee06c10413a83ab2add242d58b5/users/83a90b57c53348199830423d46cdf223/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:40.313391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60aaad-6dff-460e-9ea0-1d2942f20163 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.327667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.328128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.328406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.384323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] MFA Rules not processed for user `83a90b57c53348199830423d46cdf223`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.387834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.400774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:40.412007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.412876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.413166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.413557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.416468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.418611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.451546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 941/1912] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/users => generated 289 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:40.461865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.495480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a01e2-56f1-43ec-86c6-58b8dafe4787 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.505751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.507481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.508253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.509018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.511191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.515103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.538890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 942/1913] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:40.544326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.565287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987888a8-adc7-462f-af53-ac4fa466cbeb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.594114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.595155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.595794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.596263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.597502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.599509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.627215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 972/1914] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.645187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 943/1915] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:40.647685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.649099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.651930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.653668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.689018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22166f-0d6a-4ec4-a4ae-9c2bd55958e5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.707382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.709436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.715502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.716035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] PATH_INFO: `/v3/projects/d28a2924960b484b8a80e4d6631c1119` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.721266 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28a2924960b484b8a80e4d6631c1119)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.725802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.786718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8f66a-1afd-48fd-85f9-627c6774810a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.799212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.800681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.801436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.802013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.816621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:40.832335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.841065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.866140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28a2924960b484b8a80e4d6631c1119', 'request_id': 'req-a179af40-f7ff-47c8-b2f7-aecff3068fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:40.870241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28a2924960b484b8a80e4d6631c1119', 'request_id': 'req-a179af40-f7ff-47c8-b2f7-aecff3068fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:40.870723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 944/1916] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:40.872985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.898528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4815d-4baf-4be6-8b7b-55807d5682cb service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.899405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28a2924960b484b8a80e4d6631c1119', 'request_id': 'req-a179af40-f7ff-47c8-b2f7-aecff3068fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:40.928157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] The token cache is being invalidate because project d28a2924960b484b8a80e4d6631c1119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:40.928832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:40.930708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:40.936795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 973/1917] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:40 2022] DELETE /identity/v3/projects/d28a2924960b484b8a80e4d6631c1119 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:40.943157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.950670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.096252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac84d2-82e3-4556-9154-ed5730222cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.097173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac84d2-82e3-4556-9154-ed5730222cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.097456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac84d2-82e3-4556-9154-ed5730222cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.097732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac84d2-82e3-4556-9154-ed5730222cb4 admin admin] PATH_INFO: `/v3/projects/a4706b2d50a14933938e1c4a9076435f/users/a5d4e7f895834988aeb47c13511bcecf/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.127288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179af40-f7ff-47c8-b2f7-aecff3068fbc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.143792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.145167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.145964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac84d2-82e3-4556-9154-ed5730222cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4706b2d50a14933938e1c4a9076435f, user_id=a5d4e7f895834988aeb47c13511bcecf, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.146439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.147596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] PATH_INFO: `/v3/projects/4119c41e307448af9a0e64e1365c6ec1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.151968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac84d2-82e3-4556-9154-ed5730222cb4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.153388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4119c41e307448af9a0e64e1365c6ec1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.156702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.201621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 945/1918] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:40 2022] PUT /identity/v3/projects/a4706b2d50a14933938e1c4a9076435f/users/a5d4e7f895834988aeb47c13511bcecf/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:41.210050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4a5bd-9081-47c9-8f8e-41d9bbe3d9d5 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.210050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4a5bd-9081-47c9-8f8e-41d9bbe3d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.210050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4a5bd-9081-47c9-8f8e-41d9bbe3d9d5 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.210050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 946/1919] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:33:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:41.212392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b8b0-a00e-4cca-940a-de0a1a3d9534 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.212662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b8b0-a00e-4cca-940a-de0a1a3d9534 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.212942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b8b0-a00e-4cca-940a-de0a1a3d9534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.254111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6bb4b8b0-a00e-4cca-940a-de0a1a3d9534 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.261592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6bb4b8b0-a00e-4cca-940a-de0a1a3d9534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.265994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.301257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4119c41e307448af9a0e64e1365c6ec1', 'request_id': 'req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.317587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4119c41e307448af9a0e64e1365c6ec1', 'request_id': 'req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.328019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4119c41e307448af9a0e64e1365c6ec1', 'request_id': 'req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.360893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] The token cache is being invalidate because project 4119c41e307448af9a0e64e1365c6ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:41.361682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.363465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c207b103-cc6c-4ee6-96b4-97aaaf7fbbd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.367259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 974/1920] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:40 2022] DELETE /identity/v3/projects/4119c41e307448af9a0e64e1365c6ec1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:41.368725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.371244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.371769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.428657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 947/1921] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.436528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57274c03-9c9f-42d7-808d-6fec464f79d8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.436528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57274c03-9c9f-42d7-808d-6fec464f79d8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.436528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57274c03-9c9f-42d7-808d-6fec464f79d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.440352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] MFA Rules not processed for user `a5d4e7f895834988aeb47c13511bcecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.443220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.479732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-57274c03-9c9f-42d7-808d-6fec464f79d8 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.495468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-57274c03-9c9f-42d7-808d-6fec464f79d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.563508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 975/1922] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.563508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.563969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.580381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194526a1-38cb-4b20-86b0-2fd519dfe59b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.586667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.587660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.588017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.588316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] PATH_INFO: `/v3/users/edad1f617e6c47d08e606a5a0b42ca4b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.590060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edad1f617e6c47d08e606a5a0b42ca4b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.592108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.642668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 948/1923] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.645082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.645486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.645885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.663927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edad1f617e6c47d08e606a5a0b42ca4b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.684256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edad1f617e6c47d08e606a5a0b42ca4b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.686387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.688867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.707240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 949/1924] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.711226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.712935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.717493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.727964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edad1f617e6c47d08e606a5a0b42ca4b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.728857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edad1f617e6c47d08e606a5a0b42ca4b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.741646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edad1f617e6c47d08e606a5a0b42ca4b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:41.756824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 976/1925] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:41 2022] DELETE /identity/v3/users/edad1f617e6c47d08e606a5a0b42ca4b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:41.758585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.761328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.818854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.844279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840b8bb-9707-44d4-adf2-f6c16bf83ba1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.927865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.928940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.929257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.929699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.934526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.943579 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca4dd-e7b7-46f0-99b4-1cc3dba18b6c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.953660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.954832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.955407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.956521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.959561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.983044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.985766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.998656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 977/1926] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:41.999875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.002134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.115950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.118198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.118378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a466f7f-3b03-422d-abb0-0304b7fa7769 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.124183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.124873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.125157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.125435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.126211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:42.126524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.128364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.137984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 950/1927] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:42.160076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 978/1928] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:42.162423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.176802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5eb4b-e0e6-4653-8ab0-4cc020b253db service glance] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.186369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.187075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.187371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.187650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.188363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.190544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.214928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.217532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.244882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48682-7bfd-463a-b846-12a844f3df22 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.258620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 951/1929] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.259755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.273454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.276477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.277958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.278501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] PATH_INFO: `/v3/projects/485fa9af6d4f46e9be07f479116fb260` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.280950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=485fa9af6d4f46e9be07f479116fb260)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.286094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.318718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:42.381741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485fa9af6d4f46e9be07f479116fb260', 'request_id': 'req-d9090094-761a-48db-80ea-468a4dad04ab'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:42.392577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485fa9af6d4f46e9be07f479116fb260', 'request_id': 'req-d9090094-761a-48db-80ea-468a4dad04ab'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:42.393702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aad989-efb0-4440-aac4-b41ff30be26b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.399823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485fa9af6d4f46e9be07f479116fb260', 'request_id': 'req-d9090094-761a-48db-80ea-468a4dad04ab'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:42.419595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] The token cache is being invalidate because project 485fa9af6d4f46e9be07f479116fb260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:42.419804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:42.420414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d9090094-761a-48db-80ea-468a4dad04ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:42.426251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 979/1930] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:42 2022] DELETE /identity/v3/projects/485fa9af6d4f46e9be07f479116fb260 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:42.428928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.429999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.430290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.458820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:42.460193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.489701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:42.558844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa919ca5-2451-4320-8450-3fb9bf209d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.559697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa919ca5-2451-4320-8450-3fb9bf209d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.560239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa919ca5-2451-4320-8450-3fb9bf209d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.560762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa919ca5-2451-4320-8450-3fb9bf209d63 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.561999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa919ca5-2451-4320-8450-3fb9bf209d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.565220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa919ca5-2451-4320-8450-3fb9bf209d63 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.613290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 980/1931] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:42.616314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 952/1932] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.623754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.625154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.627798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.629771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.630388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.662386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3fa47-3f16-49cd-94ef-ac41a0cfda5d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.673132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.674145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.674905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.675572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.676550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.679230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.697463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:42.697831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 981/1933] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.699490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.710447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.723543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37343-e44c-4894-a3b0-a304bdf76c25 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.725930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 953/1934] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:42.732275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.751964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.752883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.753185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.753494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.755615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.757326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.769333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8ffbb-b15b-4bcd-a1f2-d0550e3c145b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.783467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.784223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.784638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.785149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.786272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:42.786492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.788239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.790450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 982/1935] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.796422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.817387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c63c4-87cb-4a36-8aa9-975a2eccc7c6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.827262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 954/1936] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:42.832171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.832761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350827a2-2817-4171-bf42-53e61e4031b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.834848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350827a2-2817-4171-bf42-53e61e4031b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.840219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350827a2-2817-4171-bf42-53e61e4031b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.841534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350827a2-2817-4171-bf42-53e61e4031b8 admin admin] PATH_INFO: `/v3/projects/3a812e9e7e3948beb195e859727763eb/users/70ae3fad15e54ad7bbb81aaf18d5c7a4/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.850750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545b9c-a2ba-4d77-a748-19103fdcee69 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.880122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.880814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.881675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.881675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.882090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.884109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.892438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350827a2-2817-4171-bf42-53e61e4031b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a812e9e7e3948beb195e859727763eb, user_id=70ae3fad15e54ad7bbb81aaf18d5c7a4, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.899371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350827a2-2817-4171-bf42-53e61e4031b8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.947178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 983/1937] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:42 2022] PUT /identity/v3/projects/3a812e9e7e3948beb195e859727763eb/users/70ae3fad15e54ad7bbb81aaf18d5c7a4/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:42.959394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.959881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.960231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.998337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 955/1938] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/projects => generated 407 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:43.004152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.086175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] MFA Rules not processed for user `70ae3fad15e54ad7bbb81aaf18d5c7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:43.090386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:43.301427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6746e-0372-45b9-9f9a-8494ed909ba4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.326396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.327637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.328164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.328759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.330101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.352608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 984/1939] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:43.352764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.358906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.398770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294b89-9345-4c5e-b0e4-5fd099acb2f5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.415777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.416980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.417495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.417984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.418870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.421285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.434353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 956/1940] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/users => generated 304 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:43.438594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.477944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 985/1941] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:43.479008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.492500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6934d-a06e-4f8f-bf35-9cd55409633d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.522711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:43.629708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:43.648534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.649524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.649814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.650091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.651451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.653424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.675510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135d68-57de-40d2-97a9-ecfba5d768ec admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.682993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 957/1942] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.684127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.690782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.691642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.692282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.694014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.695001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.702890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.706356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84f31-050f-4145-8117-40655e8532ee admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.712812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.714033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.714033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.714367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.715727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.717481 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.747405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 958/1943] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.748531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.766485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93daca-4f05-4f44-aab8-2503e357f237 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.773525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ef63f-8193-4485-9bf1-dd088668aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.774574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ef63f-8193-4485-9bf1-dd088668aaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.775015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ef63f-8193-4485-9bf1-dd088668aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.775317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ef63f-8193-4485-9bf1-dd088668aaaa admin admin] PATH_INFO: `/v3/projects/6705652157be4ce2815bff681a420210/users/4094eaae4b634f618aa06930a2cbd91e/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.780400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 986/1944] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/users => generated 288 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:43.782121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.806627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e889e-d149-46c5-a007-8861921cf0bf admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.814985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.815584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.815943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.816322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.817228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.825082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.830999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ef63f-8193-4485-9bf1-dd088668aaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6705652157be4ce2815bff681a420210, user_id=4094eaae4b634f618aa06930a2cbd91e, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.837719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ef63f-8193-4485-9bf1-dd088668aaaa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.840156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 987/1945] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:43.846713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.879726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200c7e6-eca0-4b07-9846-a67c55b961f6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.884612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 959/1946] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:43 2022] PUT /identity/v3/projects/6705652157be4ce2815bff681a420210/users/4094eaae4b634f618aa06930a2cbd91e/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:43.894611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.896418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.896906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.953398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] MFA Rules not processed for user `4094eaae4b634f618aa06930a2cbd91e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:43.954932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:44.060409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.060409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.061194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.061194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] PATH_INFO: `/v3/projects/ba446a096939427c85fc604a44a5467f/users/a500d579d832430ba147105231cfdcd4/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.106975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba446a096939427c85fc604a44a5467f, user_id=a500d579d832430ba147105231cfdcd4, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.109560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 960/1947] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:44.114334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.116496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.144322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9a614-827e-45db-82ea-1b5428edd92e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.144601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 988/1948] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:43 2022] PUT /identity/v3/projects/ba446a096939427c85fc604a44a5467f/users/a500d579d832430ba147105231cfdcd4/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:44.145367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.310869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.314388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.315296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.318049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.318585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.325450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.367005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:44.411454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90e422-4b50-4d2e-b8ec-3e3f4023add8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.413479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 961/1949] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/projects => generated 409 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:44.419451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.508147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:44.556712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.558570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.564565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.567742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.568752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.572387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.578138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122bf37-1d0f-4c7c-bf04-52377ff07165 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.585940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.589762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.590061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.590324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.591152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.593114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.611980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 989/1950] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:44.619190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.636734 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9b284-bc7f-4cf9-a3fb-29f868ff0d12 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.644173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf2fbe-49bc-453b-a9eb-17e31e0831ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.644569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 962/1951] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:44.648711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.649547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf2fbe-49bc-453b-a9eb-17e31e0831ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.650546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf2fbe-49bc-453b-a9eb-17e31e0831ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.650926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf2fbe-49bc-453b-a9eb-17e31e0831ef admin admin] PATH_INFO: `/v3/projects/ba446a096939427c85fc604a44a5467f/users/a500d579d832430ba147105231cfdcd4/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.654281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf2fbe-49bc-453b-a9eb-17e31e0831ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba446a096939427c85fc604a44a5467f, user_id=a500d579d832430ba147105231cfdcd4, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.661796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf2fbe-49bc-453b-a9eb-17e31e0831ef admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.673291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ec9c-0535-4eec-84c6-99f24bd398df admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.690665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 990/1952] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:44 2022] PUT /identity/v3/projects/ba446a096939427c85fc604a44a5467f/users/a500d579d832430ba147105231cfdcd4/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:44.692597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.699978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.699978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.778292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] MFA Rules not processed for user `a500d579d832430ba147105231cfdcd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:44.779652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:44.849746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.850965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.851517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.852032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.853434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.857467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.895116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 963/1953] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:44.899361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.923563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea51f1f-5b97-412a-9a36-80e1bb72502f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.944202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.944895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.945190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.945476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] PATH_INFO: `/v3/projects/243be64af38e4fdbac864344e4174548/users/de50f0739d4f40109b697e58f201a841/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.959319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 991/1954] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:44.980669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.982590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:44.999453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243be64af38e4fdbac864344e4174548, user_id=de50f0739d4f40109b697e58f201a841, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.008446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.044300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 964/1955] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:44 2022] PUT /identity/v3/projects/243be64af38e4fdbac864344e4174548/users/de50f0739d4f40109b697e58f201a841/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:45.048254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.106289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667530e-887e-4444-bb61-145f2abd2eef None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.171589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:45.177790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.178483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.178771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.179079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.188103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dba8f-eb57-4686-b0ca-ec35734fc2c6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.194964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.195824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.196151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.196642 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.197400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.200739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.214542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 965/1956] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:45.220145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:45.220568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.232226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cb001-583f-4a74-a957-e6e96d86c08e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.237382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e786691-f968-4c34-b60a-d82f492ae6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.238312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e786691-f968-4c34-b60a-d82f492ae6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.238608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e786691-f968-4c34-b60a-d82f492ae6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.238884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e786691-f968-4c34-b60a-d82f492ae6f1 admin admin] PATH_INFO: `/v3/projects/243be64af38e4fdbac864344e4174548/users/de50f0739d4f40109b697e58f201a841/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.241128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e786691-f968-4c34-b60a-d82f492ae6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243be64af38e4fdbac864344e4174548, user_id=de50f0739d4f40109b697e58f201a841, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.243946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e786691-f968-4c34-b60a-d82f492ae6f1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.266823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 966/1957] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:45 2022] PUT /identity/v3/projects/243be64af38e4fdbac864344e4174548/users/de50f0739d4f40109b697e58f201a841/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:45.272901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.273225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.273506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.274627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.278056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.298080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 992/1958] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:45.308368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] MFA Rules not processed for user `de50f0739d4f40109b697e58f201a841`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:45.309767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:45.429048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 967/1959] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:45.443014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.445439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:45.456841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.482497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c89bd-bdea-4376-9edd-f377ca500b8d service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.627465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97d99-165f-49b7-ab07-aa8721ce0685 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.642997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.644205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.644493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720b6ad-132c-47cd-a74c-b64f8ef4cb5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.645205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720b6ad-132c-47cd-a74c-b64f8ef4cb5d service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.645641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720b6ad-132c-47cd-a74c-b64f8ef4cb5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.645899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720b6ad-132c-47cd-a74c-b64f8ef4cb5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.646688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.647022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] PATH_INFO: `/v3/users/83a90b57c53348199830423d46cdf223` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.657259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a90b57c53348199830423d46cdf223)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.662653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.671920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720b6ad-132c-47cd-a74c-b64f8ef4cb5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.674291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720b6ad-132c-47cd-a74c-b64f8ef4cb5d service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.702377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 968/1960] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:45.768228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a90b57c53348199830423d46cdf223'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:45.783395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a90b57c53348199830423d46cdf223'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:45.804337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a90b57c53348199830423d46cdf223'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:45.805427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a90b57c53348199830423d46cdf223'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:45.835479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a90b57c53348199830423d46cdf223'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:45.857935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 993/1961] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:45 2022] DELETE /identity/v3/users/83a90b57c53348199830423d46cdf223 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:45.861725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.862369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.866350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.903176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:45.903176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:45.988505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.990546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:46.062777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 969/1962] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:46.075490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.078432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:46.096939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:46.154681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:46.161193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba363e-25e5-41d9-b99e-caf09639a255 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.174435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.185871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.185871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.185871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.191002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:46.285966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c882e8-7ac4-4657-9dec-3c29fd9d6282 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.299799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.303887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.305689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.306581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.345323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.350318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.418285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 970/1963] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:46.440792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.442369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.467126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 994/1964] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:47.294590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:47.301683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.328317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:47.329911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.337658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f854b-0a81-451b-93f6-e45a2fc92b9a service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:47.347097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:47.348174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.348528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.348970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] PATH_INFO: `/v3/users/70ae3fad15e54ad7bbb81aaf18d5c7a4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.350771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ae3fad15e54ad7bbb81aaf18d5c7a4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:47.353645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:47.364519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86007878-d862-4397-a54b-97c638a56f1d service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:47.393729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:47.395756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.396546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.397051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] PATH_INFO: `/v3/projects/5f080ee06c10413a83ab2add242d58b5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.405361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f080ee06c10413a83ab2add242d58b5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:47.431525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:47.521788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ae3fad15e54ad7bbb81aaf18d5c7a4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.540725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ae3fad15e54ad7bbb81aaf18d5c7a4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.542496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ae3fad15e54ad7bbb81aaf18d5c7a4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.561705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ae3fad15e54ad7bbb81aaf18d5c7a4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.565981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f080ee06c10413a83ab2add242d58b5', 'request_id': 'req-d3202673-eaee-41cc-be16-e209df2142b0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.591240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5c5ff21b-001a-4f05-8015-ad71e57cc7e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ae3fad15e54ad7bbb81aaf18d5c7a4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.592758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f080ee06c10413a83ab2add242d58b5', 'request_id': 'req-d3202673-eaee-41cc-be16-e209df2142b0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.606614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f080ee06c10413a83ab2add242d58b5', 'request_id': 'req-d3202673-eaee-41cc-be16-e209df2142b0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.606972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 971/1965] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:47 2022] DELETE /identity/v3/users/70ae3fad15e54ad7bbb81aaf18d5c7a4 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:47.617599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.618458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.619024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.631763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] The token cache is being invalidate because project 5f080ee06c10413a83ab2add242d58b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:47.632602 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.633830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3202673-eaee-41cc-be16-e209df2142b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:47.636606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 995/1966] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:47 2022] DELETE /identity/v3/projects/5f080ee06c10413a83ab2add242d58b5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:47.644634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa3bfc-ce75-4dd0-94d9-448494368fdf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.645329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa3bfc-ce75-4dd0-94d9-448494368fdf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.646524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa3bfc-ce75-4dd0-94d9-448494368fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.672907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.691371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b4fa3bfc-ce75-4dd0-94d9-448494368fdf None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.693443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b4fa3bfc-ce75-4dd0-94d9-448494368fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.698266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.824224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 996/1967] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.835593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.836294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.836768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.897659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.903680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.935935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 972/1968] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.936244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 997/1969] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.940180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:47.945219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:47.949399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.961469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e61cdf-2d17-4718-9506-98972eb997d3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:47.987239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:47.989468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:47.990259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.990710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.995127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.996072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:47.996399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:47.998140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.041262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 973/1970] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:33:48.055171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.091989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea4e85-c391-4e2c-8086-9acf41222b09 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.094500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483f13-6281-47e3-9b43-e333dce305d5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.102274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.103746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.104228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.104927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.122138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.123210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.123686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.125951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.126772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.128968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.134498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:48.138456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.143666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.164123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 998/1971] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:48.207432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 974/1972] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:48 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:48.214645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.227845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e01ae-db92-40c4-8e8a-0812c3b250c1 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.436129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.437106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.437106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.437582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.438159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.440229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.501756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 999/1973] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:48 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:48.507130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.543176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e390d-ff36-4b95-bb6e-da0e87304f06 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.564281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.567380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.568402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.568894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.574482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.583625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.628583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.637115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 975/1974] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:48.644447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fd2a-e861-4011-a3df-694910894a67 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.654543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a61d1-3ddc-416d-86e5-0ae95c93365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.655330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a61d1-3ddc-416d-86e5-0ae95c93365b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.661862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a61d1-3ddc-416d-86e5-0ae95c93365b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.661862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a61d1-3ddc-416d-86e5-0ae95c93365b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.661862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a61d1-3ddc-416d-86e5-0ae95c93365b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.661862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a61d1-3ddc-416d-86e5-0ae95c93365b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.679116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1000/1975] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:48.681125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.708006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca6da1-36dd-4470-bb98-172c93e9c75d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.718676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.720664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.722548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.722548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] PATH_INFO: `/v3/projects/097928a25b7747e48d9258afb7f49e03/users/9a6f35f938af46409d0c860534b10d61/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.772537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097928a25b7747e48d9258afb7f49e03, user_id=9a6f35f938af46409d0c860534b10d61, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.782334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.839073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.839526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.840750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.842879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 976/1976] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:48 2022] PUT /identity/v3/projects/097928a25b7747e48d9258afb7f49e03/users/9a6f35f938af46409d0c860534b10d61/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:48.895523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] MFA Rules not processed for user `9a6f35f938af46409d0c860534b10d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:48.897924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.070166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.072338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1001/1977] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:49.283030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1485e-2979-4466-9025-5a97837854d3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.302327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.303697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.304103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.304622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.321093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.327624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.333430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.341543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.350518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4ccda-fc5e-4921-baae-85cfd59d9509 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.372423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.374058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.374715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.376780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] PATH_INFO: `/v3/projects/3a812e9e7e3948beb195e859727763eb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.380517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a812e9e7e3948beb195e859727763eb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.384605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.415793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 977/1978] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:49 2022] POST /identity/v3/projects => generated 377 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:49.424457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.435571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:49.484611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a812e9e7e3948beb195e859727763eb', 'request_id': 'req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.492507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a812e9e7e3948beb195e859727763eb', 'request_id': 'req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.505365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a812e9e7e3948beb195e859727763eb', 'request_id': 'req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.523960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] The token cache is being invalidate because project 3a812e9e7e3948beb195e859727763eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:49.524329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.525039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.527687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1002/1979] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:49 2022] DELETE /identity/v3/projects/3a812e9e7e3948beb195e859727763eb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:49.536135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.540767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.557435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532059-68aa-425d-8ab3-188509ebfa63 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.607768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:49.644645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.645474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.646487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.647222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.649987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.655260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.661927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a5c2e-caf2-42a5-9fb5-31e7f3500b8c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.671940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.673969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.674550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.675042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] PATH_INFO: `/v3/users/a500d579d832430ba147105231cfdcd4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.677667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a500d579d832430ba147105231cfdcd4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.683581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.694611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 978/1980] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:49 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:49.699795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.701583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.731872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98428568-79aa-4efb-a265-7f396603e308 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.752509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.753891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.754417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.754970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.756490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.760613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.765322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a500d579d832430ba147105231cfdcd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.776453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a500d579d832430ba147105231cfdcd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.782123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 979/1981] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:49.786608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.800221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d252b97-03c6-488b-948f-3ff45ccfcb2f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.800763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a500d579d832430ba147105231cfdcd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.801978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a500d579d832430ba147105231cfdcd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.806350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350b3d4-aa49-44b0-80c8-221abd291116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.807262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350b3d4-aa49-44b0-80c8-221abd291116 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.807559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350b3d4-aa49-44b0-80c8-221abd291116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.807849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350b3d4-aa49-44b0-80c8-221abd291116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.808596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350b3d4-aa49-44b0-80c8-221abd291116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.810463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350b3d4-aa49-44b0-80c8-221abd291116 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.814119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a500d579d832430ba147105231cfdcd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:49.821394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1003/1982] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:49 2022] DELETE /identity/v3/users/a500d579d832430ba147105231cfdcd4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:49.835258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 980/1983] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:49.837223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.840209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.887380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:49.938355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d6e97-9c04-4eaf-8c88-aaa321fd45f5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.943803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.944987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.945278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.945560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] PATH_INFO: `/v3/projects/73a46e9c0885440e8d2e2dc4a6fb880e/users/fa012e84e40f42419c7bc65e50b99256/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.971020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a46e9c0885440e8d2e2dc4a6fb880e, user_id=fa012e84e40f42419c7bc65e50b99256, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.974108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.999179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1004/1984] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:49 2022] PUT /identity/v3/projects/73a46e9c0885440e8d2e2dc4a6fb880e/users/fa012e84e40f42419c7bc65e50b99256/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:50.006145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.006507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.006711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.040807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] MFA Rules not processed for user `fa012e84e40f42419c7bc65e50b99256`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:50.042459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:50.144346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 981/1985] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:50.149898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.256624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700c56a-0b8f-4f42-8e14-1ae9d91c7b44 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:50.263225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:50.263946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.264231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.264510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.267618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.271911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.330612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1005/1986] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:50 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:50.337183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.342769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.348486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:50.384068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d662b80-8d40-4100-b9c8-6559186ccd65 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:50.558802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c7dd-ccbe-4018-9cc2-094bdf37041a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:50.567609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:50.569128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.569628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.570119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] PATH_INFO: `/v3/projects/ba446a096939427c85fc604a44a5467f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.573229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba446a096939427c85fc604a44a5467f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.577026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:50.578514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.578735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.579085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.579503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.580701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.584248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.620603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 982/1987] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:50 2022] POST /identity/v3/users => generated 295 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:50.622423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.633904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220499e-822b-4fd8-beee-1be1d6f31e41 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:50.641838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:50.642575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.642857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.643166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.643942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.645730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.673933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 983/1988] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:50.676329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.681283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:50.705898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf9f1-ada6-47d3-bec8-88bc1181e7cc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:50.711349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba446a096939427c85fc604a44a5467f', 'request_id': 'req-813a4f89-9020-4f8a-8625-b280498301d9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.727517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba446a096939427c85fc604a44a5467f', 'request_id': 'req-813a4f89-9020-4f8a-8625-b280498301d9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.735442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:50.749769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba446a096939427c85fc604a44a5467f', 'request_id': 'req-813a4f89-9020-4f8a-8625-b280498301d9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.785075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] The token cache is being invalidate because project ba446a096939427c85fc604a44a5467f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:50.785774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.788127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-813a4f89-9020-4f8a-8625-b280498301d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.792984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1006/1989] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:50 2022] DELETE /identity/v3/projects/ba446a096939427c85fc604a44a5467f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:50.798280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef5613-6875-4c31-98d3-6b10d95c2c9c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.798569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef5613-6875-4c31-98d3-6b10d95c2c9c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.798856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef5613-6875-4c31-98d3-6b10d95c2c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.821258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c4ef5613-6875-4c31-98d3-6b10d95c2c9c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:50.823404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c4ef5613-6875-4c31-98d3-6b10d95c2c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:50.854736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:50.856496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.857107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.857675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] PATH_INFO: `/v3/projects/65220a3a84fe400680a95a8c0de389ba/users/350cce9f6fd74592b12efb101131771c/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.897776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65220a3a84fe400680a95a8c0de389ba, user_id=350cce9f6fd74592b12efb101131771c, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.903824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.911501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1007/1990] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:50.923518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.924120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.924695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.928638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 984/1991] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:50 2022] PUT /identity/v3/projects/65220a3a84fe400680a95a8c0de389ba/users/350cce9f6fd74592b12efb101131771c/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:50.936295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.940218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:50.968997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:50.971179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:51.078516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f9d28-fdd0-4f47-bd13-97abfba25332 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.086098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.086904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.087350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.087730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.088676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.090457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.115609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 985/1992] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:51.122599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.141191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1008/1993] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:51.147709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f91492-46f0-459c-91fc-4827093ea23f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.151168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.154469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26232597-8e49-46a5-9721-590b95cc0754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.157264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26232597-8e49-46a5-9721-590b95cc0754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.157889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26232597-8e49-46a5-9721-590b95cc0754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.158461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26232597-8e49-46a5-9721-590b95cc0754 admin admin] PATH_INFO: `/v3/projects/65220a3a84fe400680a95a8c0de389ba/users/350cce9f6fd74592b12efb101131771c/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.162607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26232597-8e49-46a5-9721-590b95cc0754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65220a3a84fe400680a95a8c0de389ba, user_id=350cce9f6fd74592b12efb101131771c, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.164753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f545-a82b-40f6-aa5d-9fe3576fb4dd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.169666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26232597-8e49-46a5-9721-590b95cc0754 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.177863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.179486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.180471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.180974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.184753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:51.184753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.187268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.199106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 986/1994] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:51 2022] PUT /identity/v3/projects/65220a3a84fe400680a95a8c0de389ba/users/350cce9f6fd74592b12efb101131771c/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:51.211175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.211667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.212203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.215640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1009/1995] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:51.221829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.245738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb19bf8-be64-44dd-ae1f-ef5fec8fda46 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.263095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] MFA Rules not processed for user `350cce9f6fd74592b12efb101131771c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:51.265146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:51.324197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:51.333034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.334882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.335355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.335635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.336389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.339904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.379853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 987/1996] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:51.389714 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1010/1997] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:51 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:51.391351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.393187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.395115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:51.409056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64270e6-2e66-476f-a3b3-eced551c5554 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.415433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:51.513835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.514689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.514877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449aeb0-a0e4-4cc7-9762-930cd019f82b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.515193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.515594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.516456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.518904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.524222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.526371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.527231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.527930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.532450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:51.567012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 988/1998] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:51 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:51.569401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.600807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac53b1-3d56-41ce-8874-007101a0b1f2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.609773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.611003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.611613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.612187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.613608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.617664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.640943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 989/1999] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:51.646945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.665734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d198fe-00fe-48e6-8386-0c0d73eeac75 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.679335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.680282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.680726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.681252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.682371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.684519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.707685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 990/2000] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:51.709900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.727518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750328d-0f88-4dcd-afec-052d6fd3a1eb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.730908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.732466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96663bc-0f09-47ab-ab5c-a287c6d7c93e service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.735499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.737767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.738496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.739089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] PATH_INFO: `/v3/projects/02e582959bf04b6fb7f41b28c4d9ea0b/users/32263acdff45493f9c89aad784c6b1b8/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.749223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1011/2001] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:51.767979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e582959bf04b6fb7f41b28c4d9ea0b, user_id=32263acdff45493f9c89aad784c6b1b8, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.771322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.796602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 991/2002] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:51 2022] PUT /identity/v3/projects/02e582959bf04b6fb7f41b28c4d9ea0b/users/32263acdff45493f9c89aad784c6b1b8/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:51.815131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.815622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.816122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.868786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] MFA Rules not processed for user `32263acdff45493f9c89aad784c6b1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:51.871511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:52.009622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1012/2003] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:52.016155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:52.156115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea16bf7-92e6-49a0-81e4-576f18cb75c1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:52.169003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:52.170337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:52.170909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:52.171482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:52.173233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:52.178210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:52.267286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 992/2004] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:52 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:52.272638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:52.327432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d700794-ba67-452d-b3f9-492bac50d475 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:52.410797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:52.477518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:52.478457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:52.479499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:52.479950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:52.481184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:52.483587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:52.516782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1013/2005] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:52 2022] POST /identity/v3/users => generated 299 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:52.521994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:52.535899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cf6a8-31cc-4164-a0b8-54d2605c0516 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:52.543181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:52.544094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:52.544511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:52.544966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:52.546065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:52.548075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:52.574486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 993/2006] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:52.579377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:52.606405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a136e-a964-41ad-a090-10ae1c7e23d2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:52.671896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:52.674082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:52.676870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:52.680620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] PATH_INFO: `/v3/projects/19ac66bacc4b4f128a34ebbfab2e1352/users/9c2ca4d78ad047c8bbe8dee1d04f5d30/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:52.771929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ac66bacc4b4f128a34ebbfab2e1352, user_id=9c2ca4d78ad047c8bbe8dee1d04f5d30, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:52.774743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:52.810908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1014/2007] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:52 2022] PUT /identity/v3/projects/19ac66bacc4b4f128a34ebbfab2e1352/users/9c2ca4d78ad047c8bbe8dee1d04f5d30/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:52.821984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:52.847296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38880d-8c7f-4c3c-af51-391c003d54fa admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:52.896872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:52.959527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5b2c-13ec-44b5-844f-25dad3273b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:52.960332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5b2c-13ec-44b5-844f-25dad3273b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:52.960707 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5b2c-13ec-44b5-844f-25dad3273b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:52.961029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5b2c-13ec-44b5-844f-25dad3273b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:52.961895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5b2c-13ec-44b5-844f-25dad3273b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:52.964342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5b2c-13ec-44b5-844f-25dad3273b35 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:52.976425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 994/2008] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:52.981411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:52.998006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec207d0-f94c-4f7d-80b2-d5f9d0070c95 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:53.003828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:53.005284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:53.005732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:53.006378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] PATH_INFO: `/v3/projects/19ac66bacc4b4f128a34ebbfab2e1352/users/9c2ca4d78ad047c8bbe8dee1d04f5d30/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:53.009748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ac66bacc4b4f128a34ebbfab2e1352, user_id=9c2ca4d78ad047c8bbe8dee1d04f5d30, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:53.013595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:53.028786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1015/2009] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:52 2022] PUT /identity/v3/projects/19ac66bacc4b4f128a34ebbfab2e1352/users/9c2ca4d78ad047c8bbe8dee1d04f5d30/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:53.043741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:53.044208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:53.044643 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:53.109011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] MFA Rules not processed for user `9c2ca4d78ad047c8bbe8dee1d04f5d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:53.111215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:53.216012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 995/2010] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:53.234156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:53.371731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa381e-f0c7-4250-8915-84ae48d3db2b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:53.380204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e18c3-b325-40a1-ad27-e050eeee6fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:53.381948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e18c3-b325-40a1-ad27-e050eeee6fde service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:53.382446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e18c3-b325-40a1-ad27-e050eeee6fde service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:53.383234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e18c3-b325-40a1-ad27-e050eeee6fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:53.408924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e18c3-b325-40a1-ad27-e050eeee6fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:53.415144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e18c3-b325-40a1-ad27-e050eeee6fde service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:53.443827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1016/2011] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:53.517278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:53.636432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1b96-b32a-42c5-9fb7-355f72b53dec None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:53.652165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:53.652165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:53.663207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:53.664053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] PATH_INFO: `/v3/users/32263acdff45493f9c89aad784c6b1b8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:53.672050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32263acdff45493f9c89aad784c6b1b8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:53.674422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:53.771495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32263acdff45493f9c89aad784c6b1b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.775262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:53.789377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32263acdff45493f9c89aad784c6b1b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.790167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32263acdff45493f9c89aad784c6b1b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.801897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32263acdff45493f9c89aad784c6b1b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.821406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32263acdff45493f9c89aad784c6b1b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.835124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 996/2012] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:53 2022] DELETE /identity/v3/users/32263acdff45493f9c89aad784c6b1b8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:53.839845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:53.840426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:53.840907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:53.888384 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:53.890177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:54.055433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1017/2013] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:54.070792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:54.073087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:54.272395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a586096-f3d5-4504-a255-ed05a25ed2f1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:54.278559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:54.284438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:54.296427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:54.297497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:54.298558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:54.299000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:54.313848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:54.315831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:54.343417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 997/2014] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:54.354506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:54.472267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b512c-db59-4b07-801c-7571d87d2f04 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:54.492622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:54.493785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:54.494330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:54.494883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:54.501098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:54.601960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:54.603534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:54.629993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1018/2015] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:54.968867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:54.973428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:55.039351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf5663-8c18-44c4-8dcd-d04d96ead736 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:55.046878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:55.050413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.050832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.051249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] PATH_INFO: `/v3/projects/02e582959bf04b6fb7f41b28c4d9ea0b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.055324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e582959bf04b6fb7f41b28c4d9ea0b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:55.058341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:55.177743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e582959bf04b6fb7f41b28c4d9ea0b', 'request_id': 'req-b880db24-a23a-483b-a17e-cbddb2a80518'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.193301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e582959bf04b6fb7f41b28c4d9ea0b', 'request_id': 'req-b880db24-a23a-483b-a17e-cbddb2a80518'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.207682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e582959bf04b6fb7f41b28c4d9ea0b', 'request_id': 'req-b880db24-a23a-483b-a17e-cbddb2a80518'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.232771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] The token cache is being invalidate because project 02e582959bf04b6fb7f41b28c4d9ea0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:55.233022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.234477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.241123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 998/2016] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:54 2022] DELETE /identity/v3/projects/02e582959bf04b6fb7f41b28c4d9ea0b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:55.242541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:55.274016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e19ece-847d-4e3c-8ca1-274e68b66b53 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:55.387469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:55.388747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.389041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.389332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] PATH_INFO: `/v3/users/9c2ca4d78ad047c8bbe8dee1d04f5d30` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.390969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c2ca4d78ad047c8bbe8dee1d04f5d30)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:55.393910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:55.486807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c2ca4d78ad047c8bbe8dee1d04f5d30'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.496078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c2ca4d78ad047c8bbe8dee1d04f5d30'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.528529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c2ca4d78ad047c8bbe8dee1d04f5d30'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.529571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c2ca4d78ad047c8bbe8dee1d04f5d30'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.542934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:55.549398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c2ca4d78ad047c8bbe8dee1d04f5d30'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.571185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1019/2017] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:55 2022] DELETE /identity/v3/users/9c2ca4d78ad047c8bbe8dee1d04f5d30 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:55.865270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:55.868028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:55.952421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880db24-a23a-483b-a17e-cbddb2a80518 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:55.962562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:56.021479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.022199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.022485 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.022790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.024968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:56.103824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.105839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.121645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 999/2018] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:56.905873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.908256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.073885 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33849173-959e-4def-a1be-30565bb756a9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.076338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:57.078837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.083528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.085388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.086161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.086864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] PATH_INFO: `/v3/projects/19ac66bacc4b4f128a34ebbfab2e1352` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.090380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ac66bacc4b4f128a34ebbfab2e1352)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:57.094500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:57.106610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4d2f3-246b-40e0-9010-ffee8bff18c6 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.115133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.117166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.117669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.118394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] PATH_INFO: `/v3/users/4094eaae4b634f618aa06930a2cbd91e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.121412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4094eaae4b634f618aa06930a2cbd91e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:57.125377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:57.210455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ac66bacc4b4f128a34ebbfab2e1352', 'request_id': 'req-5d420958-e855-47bb-9006-87d07b2b14f3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.226482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ac66bacc4b4f128a34ebbfab2e1352', 'request_id': 'req-5d420958-e855-47bb-9006-87d07b2b14f3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.228988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4094eaae4b634f618aa06930a2cbd91e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.242624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ac66bacc4b4f128a34ebbfab2e1352', 'request_id': 'req-5d420958-e855-47bb-9006-87d07b2b14f3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.242900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4094eaae4b634f618aa06930a2cbd91e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.243991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4094eaae4b634f618aa06930a2cbd91e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.255403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4094eaae4b634f618aa06930a2cbd91e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.287217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ba2b66dc-bda6-482d-99c5-02973d768ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4094eaae4b634f618aa06930a2cbd91e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.300258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] The token cache is being invalidate because project 19ac66bacc4b4f128a34ebbfab2e1352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:57.300796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.307515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d420958-e855-47bb-9006-87d07b2b14f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.308830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1000/2019] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:57 2022] DELETE /identity/v3/users/4094eaae4b634f618aa06930a2cbd91e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:57.317834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1020/2020] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:56 2022] DELETE /identity/v3/projects/19ac66bacc4b4f128a34ebbfab2e1352 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:57.318172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07127-bb7f-4b7e-97c5-e943f107d46e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.318172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07127-bb7f-4b7e-97c5-e943f107d46e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.318172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07127-bb7f-4b7e-97c5-e943f107d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.321874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.323620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.324245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.363865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5cf07127-bb7f-4b7e-97c5-e943f107d46e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:57.365130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5cf07127-bb7f-4b7e-97c5-e943f107d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.399719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:57.402063 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.437934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:57.550368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1001/2021] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:57.600988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1021/2022] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:57.609203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.609736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.610196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.611389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:57.614124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.638786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:57.640493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.673453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1002/2023] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:57.679518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:57.701859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c8a6d-8737-4741-8e95-b631cb5cb858 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.722726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.724372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:57.726531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.729278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.732047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.739203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:57.740423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:57.748630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:57.782195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767c7da-7a70-41b5-8212-92c0ee39b13e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.804437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1003/2024] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:57.811821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:57.812504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.818045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.831111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.831111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.854732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e89f-c82e-4129-993f-1751b5a2ef13 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.886507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:57.892617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:57.894224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.896188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.898664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.900455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.902792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:57.911017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:57.967814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1022/2025] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:58.026995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1004/2026] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:57 2022] POST /identity/v3/projects => generated 391 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:58.033573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.062116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752256d-1c74-4a75-b323-5e0032fa9c52 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.238748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.240004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.240872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.242026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.242609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.245772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.285676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1023/2027] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:58 2022] POST /identity/v3/users => generated 296 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:58.291339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.307786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405fbf-2d9e-49c2-8ad6-de5b01a70e3b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.315429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.315429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.315761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.316033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.316911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.318680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.336721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1005/2028] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:58.343336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.356547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adf941-20a5-48d4-97f3-dd8f8f609858 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.362061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cf15f-42a7-45f1-8c3c-c8446402b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.362776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cf15f-42a7-45f1-8c3c-c8446402b5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.363095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cf15f-42a7-45f1-8c3c-c8446402b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.363557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cf15f-42a7-45f1-8c3c-c8446402b5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.364326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cf15f-42a7-45f1-8c3c-c8446402b5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.366034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cf15f-42a7-45f1-8c3c-c8446402b5bb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.380126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1024/2029] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:58.386712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.406180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859277f-4c71-491a-a9a0-03e8b12b023f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.415476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.419091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.419751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.420274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] PATH_INFO: `/v3/projects/fda1088c14544eb8b3ffc5cf72194afa/users/25613aa4813a452c82a2cf3c3aa7fb6c/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.455267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda1088c14544eb8b3ffc5cf72194afa, user_id=25613aa4813a452c82a2cf3c3aa7fb6c, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.466223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.488924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1006/2030] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:58 2022] PUT /identity/v3/projects/fda1088c14544eb8b3ffc5cf72194afa/users/25613aa4813a452c82a2cf3c3aa7fb6c/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:58.511808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.512333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.513478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.589852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] MFA Rules not processed for user `25613aa4813a452c82a2cf3c3aa7fb6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:58.592974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:58.746290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1025/2031] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:33:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:58.754072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.893261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb983202-1312-4a1b-b870-64518771947d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.904999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.906333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.906863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.907429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.909335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.913148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:59.002547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1007/2032] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:58 2022] POST /identity/v3/projects => generated 393 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:59.004771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.029297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c7d67-7c47-46ff-aa07-7fdac4502ce2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:59.143092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:59.233867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:59.236578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:59.237307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:59.238851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:59.240409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:59.246170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:59.311692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1026/2033] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:59 2022] POST /identity/v3/users => generated 303 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:59.319115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.341463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ee19a-ed84-4335-a0a9-d600f39cb87d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:59.354321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:59.359363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:59.359363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:59.359363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:59.359363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:59.363159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:59.388615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1008/2034] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:59.393287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.412667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832ff6-0d12-4bad-92a0-7b775389b04f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:59.427802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.432371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:59.446615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:59.448786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:59.449079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:59.449361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] PATH_INFO: `/v3/projects/679fcfb255f94554b3f653cd0c03b0a8/users/4a765192dba1427d99f708048473fd2a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:59.482983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=679fcfb255f94554b3f653cd0c03b0a8, user_id=4a765192dba1427d99f708048473fd2a, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:59.488865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e366a-396b-44dd-9287-0c0b3d24968f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:59.489695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:59.512930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:59.517097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:59.521172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:59.523294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] PATH_INFO: `/v3/projects/6705652157be4ce2815bff681a420210` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:59.530039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6705652157be4ce2815bff681a420210)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:59.540582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:59.549453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1027/2035] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:59 2022] PUT /identity/v3/projects/679fcfb255f94554b3f653cd0c03b0a8/users/4a765192dba1427d99f708048473fd2a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:59.552476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.658043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:59.735897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6705652157be4ce2815bff681a420210', 'request_id': 'req-665c23e5-3793-4ae4-a2de-8bcf18466c04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:59.756859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6705652157be4ce2815bff681a420210', 'request_id': 'req-665c23e5-3793-4ae4-a2de-8bcf18466c04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:59.778278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6705652157be4ce2815bff681a420210', 'request_id': 'req-665c23e5-3793-4ae4-a2de-8bcf18466c04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:59.791193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d3ec3-e90c-4595-bda8-2d7a63f9f6c6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:59.803659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] The token cache is being invalidate because project 6705652157be4ce2815bff681a420210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:59.803969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:59.807127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:59.813594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1009/2036] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:59 2022] DELETE /identity/v3/projects/6705652157be4ce2815bff681a420210 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:59.814653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.820829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:59.907815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:59.908676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:59.910572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:59.910847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:59.911775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:59.913587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:59.937899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1028/2037] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:59.942892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:59.945794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:59.973658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbeac-abb6-41c9-ae36-285a0125cf20 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:59.987883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809bf4a-3c4e-4a13-8d27-74c240aef30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:59.989691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809bf4a-3c4e-4a13-8d27-74c240aef30f admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:59.990441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809bf4a-3c4e-4a13-8d27-74c240aef30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:59.990955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809bf4a-3c4e-4a13-8d27-74c240aef30f admin admin] PATH_INFO: `/v3/projects/679fcfb255f94554b3f653cd0c03b0a8/users/4a765192dba1427d99f708048473fd2a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:59.995749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809bf4a-3c4e-4a13-8d27-74c240aef30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=679fcfb255f94554b3f653cd0c03b0a8, user_id=4a765192dba1427d99f708048473fd2a, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:00.004686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809bf4a-3c4e-4a13-8d27-74c240aef30f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:00.028883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c23e5-3793-4ae4-a2de-8bcf18466c04 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:00.034014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1029/2038] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:59 2022] PUT /identity/v3/projects/679fcfb255f94554b3f653cd0c03b0a8/users/4a765192dba1427d99f708048473fd2a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:00.048077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:00.048583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:00.049087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:00.106002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] MFA Rules not processed for user `4a765192dba1427d99f708048473fd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:00.108118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.149835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:00.202013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:00.203595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:00.204020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:00.204311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] PATH_INFO: `/v3/users/de50f0739d4f40109b697e58f201a841` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:00.205772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de50f0739d4f40109b697e58f201a841)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:00.208723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:00.255997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1030/2039] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:00.272772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:00.280120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.310851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de50f0739d4f40109b697e58f201a841'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:00.331562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de50f0739d4f40109b697e58f201a841'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:00.332647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de50f0739d4f40109b697e58f201a841'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:00.344441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:00.350221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de50f0739d4f40109b697e58f201a841'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:00.378694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de50f0739d4f40109b697e58f201a841'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:00.391263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1010/2040] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:59 2022] DELETE /identity/v3/users/de50f0739d4f40109b697e58f201a841 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:00.422509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4c33b-8ea4-4fca-a07b-a6b8279d4a18 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:00.526104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:00.527549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:00.527999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:00.528502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:00.535606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.683593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:00.685759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:00.698469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1031/2041] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:00.762755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:00.764729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.782471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d198-c06d-42e9-9240-7dd0136216f2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:00.788477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:00.790187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:00.790187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:00.790187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:00.793468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.904237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:00.919695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ac755-7cf4-4b8b-8292-0f5113203757 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:00.964535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1011/2042] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:01.044110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:01.050331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.146367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:01.215850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96172c01-b68f-4da3-bc05-1854973dfef8 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.221653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.223578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.224538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.224869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] PATH_INFO: `/v3/projects/243be64af38e4fdbac864344e4174548` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.227457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=243be64af38e4fdbac864344e4174548)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.230404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:01.384279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243be64af38e4fdbac864344e4174548', 'request_id': 'req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.401361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243be64af38e4fdbac864344e4174548', 'request_id': 'req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.413196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243be64af38e4fdbac864344e4174548', 'request_id': 'req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.448783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] The token cache is being invalidate because project 243be64af38e4fdbac864344e4174548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:01.450354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.451864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.456139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1032/2043] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:01 2022] DELETE /identity/v3/projects/243be64af38e4fdbac864344e4174548 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:01.463864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855eef3-b7cb-4c7f-8080-29386ab17829 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.464647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855eef3-b7cb-4c7f-8080-29386ab17829 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.464829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855eef3-b7cb-4c7f-8080-29386ab17829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.516727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6855eef3-b7cb-4c7f-8080-29386ab17829 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:01.519505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6855eef3-b7cb-4c7f-8080-29386ab17829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.649356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:01.652505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.730540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1012/2044] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:01.746831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.746831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.746831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.819868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:01.824366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.855116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1013/2045] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:01.862594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:01.881392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b71b99-3ed8-4dc6-9042-0f39ce040b2e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.890751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.892978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dc908-87ba-4540-a980-19aa305d7683 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.893293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.893293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.893769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] PATH_INFO: `/v3/users/9a6f35f938af46409d0c860534b10d61` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.896572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6f35f938af46409d0c860534b10d61)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.902652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:01.909886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.911517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.912276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.913068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.914702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:01.915419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.918694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:01.944069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1014/2046] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:01.958322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:01.982114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64dc18-eecc-454b-9caf-310000215631 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.991516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.992340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.992781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.993232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.995025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.998178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.017488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6f35f938af46409d0c860534b10d61'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:02.034536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6f35f938af46409d0c860534b10d61'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:02.039535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:02.046058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1015/2047] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:01 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:02.053275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.069866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6f35f938af46409d0c860534b10d61'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:02.070878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6f35f938af46409d0c860534b10d61'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:02.084902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27bdc555-b280-48a2-b13e-0fe6d262b609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6f35f938af46409d0c860534b10d61'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:02.102132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1033/2048] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:01 2022] DELETE /identity/v3/users/9a6f35f938af46409d0c860534b10d61 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:02.105234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.105743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.106400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.144866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:02.147312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:02.195393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:02.309135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1034/2049] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:02.314772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc947d-2839-400a-8e28-e4ce6776b8fd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.323282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.325140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:02.325329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.326118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.326600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.327092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.328657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.332392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.374760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1016/2050] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:02.385032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.396847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:02.423792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911daa1-9cdf-45cf-a447-d20f31ee8f9d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.434359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.435798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.436437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.437073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.438722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.442472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.442932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70f71-1295-41e9-9e28-cd5901ba034a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.449996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e4f4c-e7f8-461a-b8eb-9373ae56fcb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.450682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4f4c-e7f8-461a-b8eb-9373ae56fcb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.450974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4f4c-e7f8-461a-b8eb-9373ae56fcb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.451568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4f4c-e7f8-461a-b8eb-9373ae56fcb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.470301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1017/2051] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:02.471436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.487868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53d4fe-3add-4878-8eed-a08a1ae4ddc8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.495332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e4f4c-e7f8-461a-b8eb-9373ae56fcb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.495889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.496768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.497249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.497683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.498612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.499666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e4f4c-e7f8-461a-b8eb-9373ae56fcb1 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.500511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.514193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1018/2052] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:02.519464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.531000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1035/2053] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:02.534715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb3851-6d06-4e4b-a166-978f829f5905 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.543729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.545388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.546315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.546910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] PATH_INFO: `/v3/projects/0dfeb0d57e87412a95540391c1738620/users/a6cb07ed8c8a4d4aa3cbb79f686729d0/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.578244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dfeb0d57e87412a95540391c1738620, user_id=a6cb07ed8c8a4d4aa3cbb79f686729d0, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.583766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.612235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1019/2054] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:02 2022] PUT /identity/v3/projects/0dfeb0d57e87412a95540391c1738620/users/a6cb07ed8c8a4d4aa3cbb79f686729d0/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:02.627291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.627830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.629251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.726919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] MFA Rules not processed for user `a6cb07ed8c8a4d4aa3cbb79f686729d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:02.740081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:02.784663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.787926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.107253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883a993-7bf8-456f-b8cf-386f1697b206 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.113203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1036/2055] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:03.114774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.130967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.135234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.136253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.136745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] PATH_INFO: `/v3/users/a5d4e7f895834988aeb47c13511bcecf` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.146014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d4e7f895834988aeb47c13511bcecf)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.150653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.161077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8064fd0-fdf1-4c33-96cc-f17d59bbcea5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.178172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.180027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.180707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.181449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.183491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.223796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.231398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:03.318502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d4e7f895834988aeb47c13511bcecf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:03.331213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1037/2056] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:03.334055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.345897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d4e7f895834988aeb47c13511bcecf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:03.347227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d4e7f895834988aeb47c13511bcecf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:03.375920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d4e7f895834988aeb47c13511bcecf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:03.426146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4a7b382b-3254-47ec-8a3a-ceb618752eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d4e7f895834988aeb47c13511bcecf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:03.429342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:03.488951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1020/2057] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:02 2022] DELETE /identity/v3/users/a5d4e7f895834988aeb47c13511bcecf => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:03.498009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.500747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.505810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.599335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:03.601628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.604101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2b0a2-4704-429c-9edc-f01495c79b46 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.834340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.835715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.836340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.836930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.839208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.844792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.846328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1021/2058] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:03.857766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.860792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.885796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe792a5a-c9c1-48ea-9836-c3b529554903 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.904174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1038/2059] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/users => generated 289 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:03.908524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.911467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.913654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.914527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.916680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.919019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] PATH_INFO: `/v3/projects/097928a25b7747e48d9258afb7f49e03` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.921695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097928a25b7747e48d9258afb7f49e03)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.929352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.067154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097928a25b7747e48d9258afb7f49e03', 'request_id': 'req-4c57b082-1f3b-4955-9331-75c87e9d3a27'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.069278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba217a1-1ed6-4aca-8cfe-0428de96badb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.090032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097928a25b7747e48d9258afb7f49e03', 'request_id': 'req-4c57b082-1f3b-4955-9331-75c87e9d3a27'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.098688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097928a25b7747e48d9258afb7f49e03', 'request_id': 'req-4c57b082-1f3b-4955-9331-75c87e9d3a27'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.113077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] The token cache is being invalidate because project 097928a25b7747e48d9258afb7f49e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:04.113379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.114094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.115889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1022/2060] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:03 2022] DELETE /identity/v3/projects/097928a25b7747e48d9258afb7f49e03 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:04.116724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.124739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.137279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:04.178160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.180076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.180950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.182046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.186829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.288823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57b082-1f3b-4955-9331-75c87e9d3a27 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.295919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.296698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.297305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.297305 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.298239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.300559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.320081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1023/2061] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:04.320081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.320504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.321702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.323022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.340273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06329-4aa9-436b-aca3-77df0fb1c45b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.345496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.346196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1039/2062] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:04.347690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.355249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.355492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.363284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] PATH_INFO: `/v3/users/fa012e84e40f42419c7bc65e50b99256` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.366172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e3c8d-a06a-4520-b8de-8c7682485c53 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.368001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa012e84e40f42419c7bc65e50b99256)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.371796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.374061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.377727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.378431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.378895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] PATH_INFO: `/v3/projects/ba03f54d916a4e349e52f3132357c1e3/users/72f7dae845b54011af603424142437ab/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.398156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:04.419961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba03f54d916a4e349e52f3132357c1e3, user_id=72f7dae845b54011af603424142437ab, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.424862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.449819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1040/2063] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:04 2022] PUT /identity/v3/projects/ba03f54d916a4e349e52f3132357c1e3/users/72f7dae845b54011af603424142437ab/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:04.454150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.484545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa012e84e40f42419c7bc65e50b99256'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.498962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa012e84e40f42419c7bc65e50b99256'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.501332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa012e84e40f42419c7bc65e50b99256'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.521455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa012e84e40f42419c7bc65e50b99256'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.556143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa012e84e40f42419c7bc65e50b99256'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:04.570555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47454b5b-bf6c-49d6-8870-55c349a5116f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.570749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1024/2064] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:04 2022] DELETE /identity/v3/users/fa012e84e40f42419c7bc65e50b99256 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:04.683088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ee2f1-a520-42e8-8ac9-0b1eb0a1b8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.683899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ee2f1-a520-42e8-8ac9-0b1eb0a1b8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.684397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ee2f1-a520-42e8-8ac9-0b1eb0a1b8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.684696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ee2f1-a520-42e8-8ac9-0b1eb0a1b8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.685618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ee2f1-a520-42e8-8ac9-0b1eb0a1b8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.687678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ee2f1-a520-42e8-8ac9-0b1eb0a1b8bc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.704976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1041/2065] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:04.710709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.715553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.740968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea89f9b-d39a-4dfd-acf7-a6301063f788 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.751269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.752992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.753881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.755046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] PATH_INFO: `/v3/projects/ba03f54d916a4e349e52f3132357c1e3/users/72f7dae845b54011af603424142437ab/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.758247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba03f54d916a4e349e52f3132357c1e3, user_id=72f7dae845b54011af603424142437ab, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.763784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.787103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1025/2066] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:04 2022] PUT /identity/v3/projects/ba03f54d916a4e349e52f3132357c1e3/users/72f7dae845b54011af603424142437ab/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:04.808186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.808665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.809196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.840842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:04.883327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] MFA Rules not processed for user `72f7dae845b54011af603424142437ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:04.884754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.929267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.932177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.041982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1042/2067] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:05.055502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f700a7-734d-43ca-a5fb-01ecc1fa3638 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.061381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.062324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.070295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.072614 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.072959 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.073290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] PATH_INFO: `/v3/projects/a4706b2d50a14933938e1c4a9076435f` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.075011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4706b2d50a14933938e1c4a9076435f)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.077490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.176168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4706b2d50a14933938e1c4a9076435f', 'request_id': 'req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.192748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4706b2d50a14933938e1c4a9076435f', 'request_id': 'req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.210595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925c2e-fa0e-4bf3-a015-ba4fd8c68e2b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.214738 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4706b2d50a14933938e1c4a9076435f', 'request_id': 'req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.246692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] The token cache is being invalidate because project a4706b2d50a14933938e1c4a9076435f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:05.247792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.248001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.255002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1026/2068] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:04 2022] DELETE /identity/v3/projects/a4706b2d50a14933938e1c4a9076435f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:05.256328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.258055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.298955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:05.336098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.337962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.339541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.340225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.344029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.412363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00bf8e-f33e-4bd9-aa5b-7a07a568a0e8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.421218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.422314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.422695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.423108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] PATH_INFO: `/v3/projects/73a46e9c0885440e8d2e2dc4a6fb880e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.424985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a46e9c0885440e8d2e2dc4a6fb880e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.428308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.476688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.479694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478c1dd-8f5d-4194-ab50-1175e2f0209d service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.486725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:05.499627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1043/2069] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:05.501284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.501703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.502079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.526990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:05.528014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a46e9c0885440e8d2e2dc4a6fb880e', 'request_id': 'req-20d2f565-e3d0-4166-8cb5-190b559336ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.528777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.546849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a46e9c0885440e8d2e2dc4a6fb880e', 'request_id': 'req-20d2f565-e3d0-4166-8cb5-190b559336ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.567187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a46e9c0885440e8d2e2dc4a6fb880e', 'request_id': 'req-20d2f565-e3d0-4166-8cb5-190b559336ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.591567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] The token cache is being invalidate because project 73a46e9c0885440e8d2e2dc4a6fb880e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:05.592078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.593250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.596042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1027/2070] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:05 2022] DELETE /identity/v3/projects/73a46e9c0885440e8d2e2dc4a6fb880e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:05.602757 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.608919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.674286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1044/2071] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:05.675910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.678270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.763643 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2f565-e3d0-4166-8cb5-190b559336ab admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.774231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.777203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.777841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.778251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] PATH_INFO: `/v3/users/350cce9f6fd74592b12efb101131771c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.780665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=350cce9f6fd74592b12efb101131771c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.785352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.789801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb226eaa-0e78-445e-8194-486b31258e02 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.797386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.798563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.799035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.799510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.803588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.858060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:05.872014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350cce9f6fd74592b12efb101131771c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.885341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350cce9f6fd74592b12efb101131771c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.886476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350cce9f6fd74592b12efb101131771c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.904501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350cce9f6fd74592b12efb101131771c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.908193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.910779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818becb-3f6b-495f-81bb-5c61562d785d service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.928046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1045/2072] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:05.929203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350cce9f6fd74592b12efb101131771c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:05.930524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.931801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.933393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.942750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1028/2073] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:05 2022] DELETE /identity/v3/users/350cce9f6fd74592b12efb101131771c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:05.970755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:05.972284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:06.053012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:06.076751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1046/2074] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:06.081619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.106421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38664d8-09ee-4281-8b5a-6807482a9343 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.115606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.117602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.120418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.121190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.122787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:06.123710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.128134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.158628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1029/2075] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:06.168540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.201271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62301-48ed-40de-a4e8-d060e9b11fea None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.214431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.215705 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.216221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.216716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.218324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.223742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.270957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.279847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1047/2076] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:06 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:06.281433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:06.281774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.324518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb43af-f6af-44b0-83f5-ef19ecf12347 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.452736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89e45d-7a20-4e50-9aca-8d8c85efbf9f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.467811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.469306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.469586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.469887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.472345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.473724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.473909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.474334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.474832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] PATH_INFO: `/v3/users/25613aa4813a452c82a2cf3c3aa7fb6c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.477401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.478720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=25613aa4813a452c82a2cf3c3aa7fb6c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.487278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.523007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1048/2077] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:06 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:06.525280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.542413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720a075-df86-41f8-b396-a5c14461d49d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.551712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.552886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.554300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.554300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.555909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.562550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.592771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1049/2078] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:06.594838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.600895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25613aa4813a452c82a2cf3c3aa7fb6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:06.613085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25613aa4813a452c82a2cf3c3aa7fb6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:06.614246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25613aa4813a452c82a2cf3c3aa7fb6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:06.616038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91859b-22bc-456a-915b-858f9daea93b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.628220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.628512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25613aa4813a452c82a2cf3c3aa7fb6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:06.630114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.630927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.631813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.633950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.638467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.682344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21eebbbd-4856-4fb7-8022-123808de4e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25613aa4813a452c82a2cf3c3aa7fb6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:06.698216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1030/2079] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:06 2022] DELETE /identity/v3/users/25613aa4813a452c82a2cf3c3aa7fb6c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:06.704551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.704978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.705483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.717701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1050/2080] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:06.720211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.724559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:06.755438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:06.756953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:06.774239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df6342-0cc3-4f16-9611-18c166e877ab admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.790648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:06.881651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1031/2081] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:06.894772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.897511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:06.925852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.929626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.930178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.930586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] PATH_INFO: `/v3/projects/a2632b16449645b1b031a66d2ac250be/users/4b5a5f6a431148979b9278b6dd9dcb04/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.965724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2632b16449645b1b031a66d2ac250be, user_id=4b5a5f6a431148979b9278b6dd9dcb04, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.969031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.000691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1051/2082] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:06 2022] PUT /identity/v3/projects/a2632b16449645b1b031a66d2ac250be/users/4b5a5f6a431148979b9278b6dd9dcb04/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:07.004445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.005101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af00dea-2dd3-4dad-bf66-c92c0e1ee5dc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.008388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.048851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:07.101433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11659595-b5b3-4fbd-a485-b53fb3253137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.102785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11659595-b5b3-4fbd-a485-b53fb3253137 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.104082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11659595-b5b3-4fbd-a485-b53fb3253137 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.104634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11659595-b5b3-4fbd-a485-b53fb3253137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.107952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14628e3e-3ee4-41ad-a337-c3c89d79ae10 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.113991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.114950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.115298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.115569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] PATH_INFO: `/v3/projects/65220a3a84fe400680a95a8c0de389ba` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.117043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65220a3a84fe400680a95a8c0de389ba)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.120144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.126085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11659595-b5b3-4fbd-a485-b53fb3253137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.129151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11659595-b5b3-4fbd-a485-b53fb3253137 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.150335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1032/2083] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:07.153323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.153901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.154447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.200999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65220a3a84fe400680a95a8c0de389ba', 'request_id': 'req-6f0e7d46-73e2-450a-b36e-91f60f903e10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:07.212411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65220a3a84fe400680a95a8c0de389ba', 'request_id': 'req-6f0e7d46-73e2-450a-b36e-91f60f903e10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:07.223205 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65220a3a84fe400680a95a8c0de389ba', 'request_id': 'req-6f0e7d46-73e2-450a-b36e-91f60f903e10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:07.243508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] MFA Rules not processed for user `4b5a5f6a431148979b9278b6dd9dcb04`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:07.247316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] The token cache is being invalidate because project 65220a3a84fe400680a95a8c0de389ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:07.247525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.248622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:07.257535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f0e7d46-73e2-450a-b36e-91f60f903e10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:07.263207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1052/2084] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:07 2022] DELETE /identity/v3/projects/65220a3a84fe400680a95a8c0de389ba => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:07.270032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800764b-b6cc-436e-917b-ca2b5252049e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.270469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800764b-b6cc-436e-917b-ca2b5252049e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.271095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800764b-b6cc-436e-917b-ca2b5252049e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.315521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b800764b-b6cc-436e-917b-ca2b5252049e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:07.318496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b800764b-b6cc-436e-917b-ca2b5252049e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.444780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1053/2085] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:07.449552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.450808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.451578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.485821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1033/2086] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:07.501819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:07.504874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.505033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.506040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.526674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1054/2087] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:07.534009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.539753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:07.572129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4ad72-a3d6-49e3-b5b0-305bd2e60152 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.585189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.586088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.586380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.586660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.588069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:07.588387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.592299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.625184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1055/2088] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:07.631186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.653518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3757b-2a57-4bad-880c-19627d876ed5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.675929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.676909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.677230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.677570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.678298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.680397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.734335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f33cfe-a9d7-4e64-800d-5e3f57b9cc93 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.743298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.752500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.752763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.753249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.786234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1056/2089] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:07 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:07.788023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.797006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.807564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.810495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.830902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:07.857300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1034/2090] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:07.859025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.897875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2256d-fd2c-48df-a859-a731af6aa39e service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.026104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7324b-c59d-4d28-a51f-bfb9a55db198 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.044468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.046281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.046960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.048567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] PATH_INFO: `/v3/projects/fda1088c14544eb8b3ffc5cf72194afa` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.051041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda1088c14544eb8b3ffc5cf72194afa)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.058557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.066431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.067294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.067586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.067867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.068584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.070798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.100214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1035/2091] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:07 2022] POST /identity/v3/users => generated 286 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:08.107567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:08.122460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee992c-0257-4982-a5c8-98350897dd8f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.131510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.132326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.132584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.133766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.134733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.137918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.147564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda1088c14544eb8b3ffc5cf72194afa', 'request_id': 'req-d8892867-f60d-4080-96dd-28ed7a5f0f96'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.155368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda1088c14544eb8b3ffc5cf72194afa', 'request_id': 'req-d8892867-f60d-4080-96dd-28ed7a5f0f96'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.163169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda1088c14544eb8b3ffc5cf72194afa', 'request_id': 'req-d8892867-f60d-4080-96dd-28ed7a5f0f96'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.164405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1036/2092] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:08.166350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:08.187783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ce0f0-dcde-4743-89b1-a8a48b6d78d9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.188730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] The token cache is being invalidate because project fda1088c14544eb8b3ffc5cf72194afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:08.189098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.190683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.199141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1057/2093] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:07 2022] DELETE /identity/v3/projects/fda1088c14544eb8b3ffc5cf72194afa => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:08.201655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:08.203682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:08.338441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892867-f60d-4080-96dd-28ed7a5f0f96 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.345113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.346842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.347373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.347670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] PATH_INFO: `/v3/users/4a765192dba1427d99f708048473fd2a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.349670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a765192dba1427d99f708048473fd2a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.352589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.421788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.423040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.423701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.424265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.425753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.430535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.452606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1037/2094] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:08.458577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:08.461299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:08.472573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:08.479961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736d4b-de46-4bf4-b31a-a63d7963f43a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.481866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a765192dba1427d99f708048473fd2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.485874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07108a-350c-4898-8bb2-9ce50eca2ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.486903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07108a-350c-4898-8bb2-9ce50eca2ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.488215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07108a-350c-4898-8bb2-9ce50eca2ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.488402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07108a-350c-4898-8bb2-9ce50eca2ad4 admin admin] PATH_INFO: `/v3/projects/b52981ec207b4fd2b29b2a7cad7af0f6/users/3d5ed6131771464d9557dd595da1018a/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.506497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a765192dba1427d99f708048473fd2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.513895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07108a-350c-4898-8bb2-9ce50eca2ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52981ec207b4fd2b29b2a7cad7af0f6, user_id=3d5ed6131771464d9557dd595da1018a, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.517519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07108a-350c-4898-8bb2-9ce50eca2ad4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.541243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1038/2095] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:08 2022] PUT /identity/v3/projects/b52981ec207b4fd2b29b2a7cad7af0f6/users/3d5ed6131771464d9557dd595da1018a/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:08.554958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a765192dba1427d99f708048473fd2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.555164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.555620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.555823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a765192dba1427d99f708048473fd2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.559668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.571641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:08.578247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a765192dba1427d99f708048473fd2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.599879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1058/2096] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:08 2022] DELETE /identity/v3/users/4a765192dba1427d99f708048473fd2a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:08.601925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] MFA Rules not processed for user `3d5ed6131771464d9557dd595da1018a`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:08.604122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:08.815806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1039/2097] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:08.830095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:08.832045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:08.953968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1735e-b30d-4a66-b5ac-0420e0cc73ae admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.961427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b553-812c-4319-909e-c017096c4902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.964082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b553-812c-4319-909e-c017096c4902 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.964653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b553-812c-4319-909e-c017096c4902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.965173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b553-812c-4319-909e-c017096c4902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.993050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b553-812c-4319-909e-c017096c4902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.993050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b553-812c-4319-909e-c017096c4902 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.999702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1059/2098] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:09.287196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:09.288943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.445500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5a3a1-4cd0-4af7-bff4-539b102e9c30 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:09.456161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:09.458368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.459194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.459743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] PATH_INFO: `/v3/projects/679fcfb255f94554b3f653cd0c03b0a8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.461712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] RBAC: Authorizing `identity:delete_project(project_id=679fcfb255f94554b3f653cd0c03b0a8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:09.465397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:09.566118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '679fcfb255f94554b3f653cd0c03b0a8', 'request_id': 'req-41009915-f480-419f-99ad-ba45e1cb23df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:09.617622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '679fcfb255f94554b3f653cd0c03b0a8', 'request_id': 'req-41009915-f480-419f-99ad-ba45e1cb23df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:09.632070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '679fcfb255f94554b3f653cd0c03b0a8', 'request_id': 'req-41009915-f480-419f-99ad-ba45e1cb23df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:09.658652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] The token cache is being invalidate because project 679fcfb255f94554b3f653cd0c03b0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:09.658652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:09.661408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-41009915-f480-419f-99ad-ba45e1cb23df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:09.668474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1040/2099] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:09 2022] DELETE /identity/v3/projects/679fcfb255f94554b3f653cd0c03b0a8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:09.675861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.676630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.676630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.714851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:09.716207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.830959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1060/2100] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:09.844171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963b38-686c-41ed-b587-8603c136382e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.844751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963b38-686c-41ed-b587-8603c136382e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.845174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963b38-686c-41ed-b587-8603c136382e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.888352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-af963b38-686c-41ed-b587-8603c136382e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:09.890505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-af963b38-686c-41ed-b587-8603c136382e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.908508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1041/2101] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:09.919150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:09.953673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c6b06-d807-41e0-8185-dc54bd9be60c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:09.967819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:09.969393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.970013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.970622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.972306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:09.972968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:09.976362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.008250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1061/2102] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:10.016039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963b38-686c-41ed-b587-8603c136382e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.056501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963b38-686c-41ed-b587-8603c136382e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.072517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.073915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.074617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.075343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.077634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.081807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.128419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.169913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1042/2103] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:10 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:10.175569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.194467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a7858-2a85-4688-a717-bcd6d5e7db26 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.294457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.327405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.334010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.335530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.336397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.339649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.345261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.405131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1062/2104] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:10 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:10.411164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.425082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63117593-c4cb-4310-a2ec-facedb916266 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.436752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.437824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.438206 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.438761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.439770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.441729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.455296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1043/2105] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:10.459802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.472028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15345613-79c1-4d1f-8d2e-06d85fabda7f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.477849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27baa75-2901-4657-a112-bea3d45e5afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.479048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baa75-2901-4657-a112-bea3d45e5afd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.479375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baa75-2901-4657-a112-bea3d45e5afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.479729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27baa75-2901-4657-a112-bea3d45e5afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.480703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27baa75-2901-4657-a112-bea3d45e5afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.482552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27baa75-2901-4657-a112-bea3d45e5afd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.498412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1063/2106] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:10.500364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.513157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303d31b-061d-4fa4-84bc-cda1a4bf5a16 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.518993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.519944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.520236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.521123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] PATH_INFO: `/v3/projects/103f3495c3844c8db2f60543bbcb73a1/users/47f29ba2bed04cf5b79bc45a99fb567a/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.539393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=103f3495c3844c8db2f60543bbcb73a1, user_id=47f29ba2bed04cf5b79bc45a99fb567a, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.542937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.559037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1044/2107] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:10 2022] PUT /identity/v3/projects/103f3495c3844c8db2f60543bbcb73a1/users/47f29ba2bed04cf5b79bc45a99fb567a/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:10.572907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.573691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.573983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.611715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] MFA Rules not processed for user `47f29ba2bed04cf5b79bc45a99fb567a`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:10.613145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:10.705066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1064/2108] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:10.709649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.821086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f4533-70f9-4035-adab-6214045f5acf admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.829109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.829892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.830238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.830548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.831312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.833448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.888645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1045/2109] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:10 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:10.889679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.903541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34c884-6cac-4242-983c-1118eb18b260 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.972301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.973010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.973319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.973600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.974329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.976460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.005940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1065/2110] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:10 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:11.007302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.021978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2ed-2546-4f56-992c-5f1abfd20c4e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.028292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.029129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.029636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.029999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.030949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.033237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.053990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1046/2111] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:11.055245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.067638 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a70c5-8452-4837-8551-78b05421a1ba admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.075698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.076727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.077011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.077283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] PATH_INFO: `/v3/projects/107186a562dc4472a981b9e36393ed06/users/99f9e1e1e9ad4503874321aef571dad8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.096937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107186a562dc4472a981b9e36393ed06, user_id=99f9e1e1e9ad4503874321aef571dad8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.100244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.138788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1066/2112] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:11 2022] PUT /identity/v3/projects/107186a562dc4472a981b9e36393ed06/users/99f9e1e1e9ad4503874321aef571dad8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:11.143231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.156874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25817e6c-38cf-48e6-b6ff-9e48487266f9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.273932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e5e78-8087-4901-9a23-59361b3f8f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.275243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e5e78-8087-4901-9a23-59361b3f8f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.275532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e5e78-8087-4901-9a23-59361b3f8f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.275805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e5e78-8087-4901-9a23-59361b3f8f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.276553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e5e78-8087-4901-9a23-59361b3f8f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.278212 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e5e78-8087-4901-9a23-59361b3f8f73 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.294453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1047/2113] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:11.296161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.308891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f882e8-3294-411b-8f53-653a8012e805 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.315308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.316800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.317434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.317910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] PATH_INFO: `/v3/projects/107186a562dc4472a981b9e36393ed06/users/99f9e1e1e9ad4503874321aef571dad8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.320911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=107186a562dc4472a981b9e36393ed06, user_id=99f9e1e1e9ad4503874321aef571dad8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.324875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.337597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1067/2114] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:11 2022] PUT /identity/v3/projects/107186a562dc4472a981b9e36393ed06/users/99f9e1e1e9ad4503874321aef571dad8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:11.348570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.348708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.348926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.382264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] MFA Rules not processed for user `99f9e1e1e9ad4503874321aef571dad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:11.383717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:11.407606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:11.467116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1048/2115] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:11.480154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.482265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:11.499483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:11.555437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a446b9-eedb-49d1-a522-8eda1c15b7ea admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.560279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf844fe-e386-4387-b713-b6a9d3bb8e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.560887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf844fe-e386-4387-b713-b6a9d3bb8e7f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.561372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf844fe-e386-4387-b713-b6a9d3bb8e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.561656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf844fe-e386-4387-b713-b6a9d3bb8e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.628980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf844fe-e386-4387-b713-b6a9d3bb8e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.630698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf844fe-e386-4387-b713-b6a9d3bb8e7f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.648198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1068/2116] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:11.808344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.810181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:11.939792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ab38-8445-4708-a9f6-023e90ed36d7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.949828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.951208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.951723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.952165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] PATH_INFO: `/v3/users/a6cb07ed8c8a4d4aa3cbb79f686729d0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.954705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6cb07ed8c8a4d4aa3cbb79f686729d0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.957105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:12.077288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6cb07ed8c8a4d4aa3cbb79f686729d0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.094964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6cb07ed8c8a4d4aa3cbb79f686729d0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.096455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6cb07ed8c8a4d4aa3cbb79f686729d0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.121429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6cb07ed8c8a4d4aa3cbb79f686729d0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.154936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6cb07ed8c8a4d4aa3cbb79f686729d0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.170514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1049/2117] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:11 2022] DELETE /identity/v3/users/a6cb07ed8c8a4d4aa3cbb79f686729d0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:12.176294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:12.177163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:12.177332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:12.202017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:12.203451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:12.406700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1069/2118] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:12.433164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:12.437311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:12.543924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:12.590498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898449e-b3e3-4ffb-a2c8-7a39d81a90ac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:12.598098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:12.599311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:12.600282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:12.600282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:12.622908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:12.626180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:12.648499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1050/2119] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:13.213566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.215874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.237831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd243b0a-ccf2-400b-bd51-e7f4a40512b1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.246377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.249401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.250066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.262226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] PATH_INFO: `/v3/projects/0dfeb0d57e87412a95540391c1738620` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.267646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dfeb0d57e87412a95540391c1738620)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:13.273059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:13.376959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dfeb0d57e87412a95540391c1738620', 'request_id': 'req-0706651e-7665-4c97-ad63-558ff51f4a22'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.392121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dfeb0d57e87412a95540391c1738620', 'request_id': 'req-0706651e-7665-4c97-ad63-558ff51f4a22'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.405137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:13.409401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dfeb0d57e87412a95540391c1738620', 'request_id': 'req-0706651e-7665-4c97-ad63-558ff51f4a22'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.444594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] The token cache is being invalidate because project 0dfeb0d57e87412a95540391c1738620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:13.445500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.449741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.454302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1070/2120] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:13 2022] DELETE /identity/v3/projects/0dfeb0d57e87412a95540391c1738620 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:13.459699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.484233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42df43-c365-46c8-b1f5-9efcaf98aadd service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.627423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.628581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.643436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.643436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] PATH_INFO: `/v3/users/72f7dae845b54011af603424142437ab` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.643436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f7dae845b54011af603424142437ab)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:13.643436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:13.802947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f7dae845b54011af603424142437ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.840467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f7dae845b54011af603424142437ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.844841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f7dae845b54011af603424142437ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.870039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f7dae845b54011af603424142437ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.889159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fe0dbc53-8f13-4b7b-81dc-0d9b144c9839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f7dae845b54011af603424142437ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:13.899473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1051/2121] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:13 2022] DELETE /identity/v3/users/72f7dae845b54011af603424142437ab => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:14.897546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.900584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:14.963096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706651e-7665-4c97-ad63-558ff51f4a22 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.127348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.128572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.129090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.129645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] PATH_INFO: `/v3/projects/ba03f54d916a4e349e52f3132357c1e3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.133430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba03f54d916a4e349e52f3132357c1e3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.139087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.300834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba03f54d916a4e349e52f3132357c1e3', 'request_id': 'req-2d463385-19f7-4b9f-a554-73fe25818c35'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.314961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba03f54d916a4e349e52f3132357c1e3', 'request_id': 'req-2d463385-19f7-4b9f-a554-73fe25818c35'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.330758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba03f54d916a4e349e52f3132357c1e3', 'request_id': 'req-2d463385-19f7-4b9f-a554-73fe25818c35'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.382761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] The token cache is being invalidate because project ba03f54d916a4e349e52f3132357c1e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:15.383573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.386997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d463385-19f7-4b9f-a554-73fe25818c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.391291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1071/2122] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:14 2022] DELETE /identity/v3/projects/ba03f54d916a4e349e52f3132357c1e3 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:15.398672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.399407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.399744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.438918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:15.441034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.569349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:15.591236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1052/2123] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:15.605990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543323f5-370c-447d-baea-f216164b1bba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.606293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543323f5-370c-447d-baea-f216164b1bba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.606582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543323f5-370c-447d-baea-f216164b1bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.672479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-543323f5-370c-447d-baea-f216164b1bba None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:15.673762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-543323f5-370c-447d-baea-f216164b1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.686289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1072/2124] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:15.692693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.706310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547357d8-4aa3-4423-b595-084fe269ccc7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.723651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.724556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.724967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.725352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.726281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:15.726701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.732515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.778010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1053/2125] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:15.779169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543323f5-370c-447d-baea-f216164b1bba None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.794434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543323f5-370c-447d-baea-f216164b1bba None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.801282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.804463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.804824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.805217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.806703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.826947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.890803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1073/2126] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:15.899249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.916016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dfc1f-bc09-4a01-b5e0-a9ee840ac42a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.076442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.077766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.079275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.079794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.081355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.085677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.139512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1054/2127] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:16.140058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.153898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73732bcb-d6fb-44cd-ac7b-775692271d56 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.159250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.160273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.160753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.161385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.162635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.178540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.212997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1074/2128] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:16.219247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.268323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed98eaa-4736-4b01-a470-44f1aded8056 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.284577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e02c-4f49-488c-8c69-fdb02fcc5afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.287403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e02c-4f49-488c-8c69-fdb02fcc5afe admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.287955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e02c-4f49-488c-8c69-fdb02fcc5afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.289294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e02c-4f49-488c-8c69-fdb02fcc5afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.295366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e02c-4f49-488c-8c69-fdb02fcc5afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.297420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e02c-4f49-488c-8c69-fdb02fcc5afe admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.324855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1055/2129] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:16.331005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.347681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473541-9b85-4773-b283-a1cb44519ad2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.361264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.362274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.362572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.362901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] PATH_INFO: `/v3/projects/de144243636342679942e24260c0339b/users/6123610615a64728bf9389937d46f0df/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.388574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de144243636342679942e24260c0339b, user_id=6123610615a64728bf9389937d46f0df, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.391548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.408980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1075/2130] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:16 2022] PUT /identity/v3/projects/de144243636342679942e24260c0339b/users/6123610615a64728bf9389937d46f0df/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:16.433445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.434386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.434847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.475620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] MFA Rules not processed for user `6123610615a64728bf9389937d46f0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:16.477411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:16.625978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1056/2131] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:16.654128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.656210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:16.689195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:16.756559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575992f-3be1-4963-8f1f-179eb6a14ee9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.764896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3538c76-9182-456a-bd91-bcc294b17764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.765965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538c76-9182-456a-bd91-bcc294b17764 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.766462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538c76-9182-456a-bd91-bcc294b17764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.766930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538c76-9182-456a-bd91-bcc294b17764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.786565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3538c76-9182-456a-bd91-bcc294b17764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.788200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3538c76-9182-456a-bd91-bcc294b17764 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.801940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1076/2132] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:16.830671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.988200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:17.012357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c3b0d-a086-4bc9-acc9-427ff0ecb0d7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.031284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.031284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.031284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.031284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] PATH_INFO: `/v3/users/6123610615a64728bf9389937d46f0df` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.031765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6123610615a64728bf9389937d46f0df)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.037638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.177573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6123610615a64728bf9389937d46f0df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:17.192975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6123610615a64728bf9389937d46f0df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:17.193870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6123610615a64728bf9389937d46f0df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:17.209052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6123610615a64728bf9389937d46f0df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:17.236780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6123610615a64728bf9389937d46f0df'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:17.252530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1057/2133] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:16 2022] DELETE /identity/v3/users/6123610615a64728bf9389937d46f0df => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:17.260237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.260865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.261488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.358403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:17.360377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.472002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1077/2134] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:17.484885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.487472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.654134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.660647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c970cf6-986a-49f8-afa9-6b04060783b3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.665653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.668434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.669538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.670075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.670507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.703458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.708685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.736809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1058/2135] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:17.848718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4471a-4631-4274-9979-559021aafaf7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.857201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.858850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.859405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.859830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.862326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.093704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.108457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296ee82-14e4-40dd-b672-a360f84ce3a8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.145938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1078/2136] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:18.206866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.209551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.251384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff0bb5-c36d-4745-a378-26f890176d8f service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.271502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.272493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.272775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.273052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] PATH_INFO: `/v3/projects/de144243636342679942e24260c0339b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.280182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de144243636342679942e24260c0339b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.282605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.440678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de144243636342679942e24260c0339b', 'request_id': 'req-95a36a30-482b-4e33-b76f-d4dd76ea075a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.449636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de144243636342679942e24260c0339b', 'request_id': 'req-95a36a30-482b-4e33-b76f-d4dd76ea075a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.459906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de144243636342679942e24260c0339b', 'request_id': 'req-95a36a30-482b-4e33-b76f-d4dd76ea075a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.480144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:18.487196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] The token cache is being invalidate because project de144243636342679942e24260c0339b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:18.487899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.488515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-95a36a30-482b-4e33-b76f-d4dd76ea075a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.491662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1059/2137] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:18 2022] DELETE /identity/v3/projects/de144243636342679942e24260c0339b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:18.499031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.499666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.499789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.523158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:18.525505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.531290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:18.645374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1079/2138] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:18.663303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.663456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.663736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.722392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:18.729351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.756778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1060/2139] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:18.759259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.770271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbe4ff-3aa7-4c81-bedb-f342eef058d2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.777351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.778829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.779373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.779860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.780826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:18.781134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.782858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.803367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1080/2140] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:18.810139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.823893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7dd8c-a47d-4199-92e3-fdf5b3eb45ec None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.834082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.835201 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.835665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.836118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.838107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.842216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.895910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1061/2141] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:18 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:18.902284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.918078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49185ee9-3ba0-4039-bd73-aaf45dcedc9a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.033100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4a4b3-aa6c-4631-98fc-28763d5fe646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.033797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a4b3-aa6c-4631-98fc-28763d5fe646 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.034203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a4b3-aa6c-4631-98fc-28763d5fe646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.034595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4a4b3-aa6c-4631-98fc-28763d5fe646 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.035580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a4b3-aa6c-4631-98fc-28763d5fe646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.039471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4a4b3-aa6c-4631-98fc-28763d5fe646 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.085717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1081/2142] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:18 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:19.093369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.109478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c656-23d9-448e-b8e6-3fef04f57e02 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.109982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c656-23d9-448e-b8e6-3fef04f57e02 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.110461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c656-23d9-448e-b8e6-3fef04f57e02 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.112562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1082/2143] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:34:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:19.116060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8ab00-31e7-4167-9c04-0106dfbb65c6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.123217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.124253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.124721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.125172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.126191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.129198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.134352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a08d-6c8a-4d51-bb2b-4bc3d22b32c5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.134914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a08d-6c8a-4d51-bb2b-4bc3d22b32c5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.135441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a08d-6c8a-4d51-bb2b-4bc3d22b32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.147931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1062/2144] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:19.154216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.168554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47270d8b-0045-41cf-bf60-151d9a8e50a9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.175724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.176430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.176712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.176988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.177999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.179703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.200954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1063/2145] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:19.202493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.203693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9731a08d-6c8a-4d51-bb2b-4bc3d22b32c5 None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:19.210919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9731a08d-6c8a-4d51-bb2b-4bc3d22b32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.218682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a30d5-25d7-4653-8fc7-32e8d7973675 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.225404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.226711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.227089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.227587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] PATH_INFO: `/v3/projects/001b3b31ec1d40f0b2d50f82d1267f6b/users/a93cdd01cfed4fccb8f6571fe12692b5/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.253479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001b3b31ec1d40f0b2d50f82d1267f6b, user_id=a93cdd01cfed4fccb8f6571fe12692b5, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.258330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.297087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1064/2146] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:19 2022] PUT /identity/v3/projects/001b3b31ec1d40f0b2d50f82d1267f6b/users/a93cdd01cfed4fccb8f6571fe12692b5/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:19.297989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.299875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.315561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1083/2147] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:19.322143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.322687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.323392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.366092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] MFA Rules not processed for user `a93cdd01cfed4fccb8f6571fe12692b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:19.368525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.382360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:19.406463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04c5c-f8f9-4d0b-bcd0-d8de646f5b57 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.413961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.415966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.416308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.416821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] PATH_INFO: `/v3/users/2c07a46f581b4bbb97ad99005471ebe3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.418299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c07a46f581b4bbb97ad99005471ebe3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.420586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.455960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1084/2148] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:19.462666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.462666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.482277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c07a46f581b4bbb97ad99005471ebe3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.492134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c07a46f581b4bbb97ad99005471ebe3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.494680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c07a46f581b4bbb97ad99005471ebe3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.508612 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c07a46f581b4bbb97ad99005471ebe3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.535434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c07a46f581b4bbb97ad99005471ebe3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.550434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1065/2149] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:19 2022] DELETE /identity/v3/users/2c07a46f581b4bbb97ad99005471ebe3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:19.553888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.558470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.568600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:19.630847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83471fd7-2cad-4902-a586-2d5a8359c6c4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.677155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be3f1a-3bc2-4974-a64d-1f2f0c1aee05 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.683427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.684122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.684411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.684840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.687399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.768970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.770719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.771503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.772779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.776721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.792916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.794926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.819771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1085/2150] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:19.823942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.825545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde838a3-6053-4b34-a55e-5f209f296620 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.837563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1066/2151] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:20.242477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.242893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.427670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:20.481099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1875-ad59-4e98-9fe0-889f0080e0bc service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.492202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.495112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.495645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.496189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] PATH_INFO: `/v3/projects/7f7d739ba4574964821462bf4be0e1f5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.501511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7d739ba4574964821462bf4be0e1f5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.505633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.584515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7d739ba4574964821462bf4be0e1f5', 'request_id': 'req-51cecb0b-765b-4807-90a3-c299d750a438'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:20.593028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7d739ba4574964821462bf4be0e1f5', 'request_id': 'req-51cecb0b-765b-4807-90a3-c299d750a438'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:20.599475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7d739ba4574964821462bf4be0e1f5', 'request_id': 'req-51cecb0b-765b-4807-90a3-c299d750a438'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:20.616489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] The token cache is being invalidate because project 7f7d739ba4574964821462bf4be0e1f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:20.616865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:20.617680 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-51cecb0b-765b-4807-90a3-c299d750a438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:20.619736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1086/2152] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:20 2022] DELETE /identity/v3/projects/7f7d739ba4574964821462bf4be0e1f5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:20.625840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.626923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.626923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.648206 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:20.650928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.725787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1067/2153] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:20.737561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.738465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.738972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.772356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:20.774996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.789181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1087/2154] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:20.795555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.814416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847e54c-a7a2-434b-a1fc-84000abb58fc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.822649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.823379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.823675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.823954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.824710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:20.825023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.826725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.840525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1068/2155] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:20.845741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.858048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3178-ebc0-4081-afb8-2f0155c4b314 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.864307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.865334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.865682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.866009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.867248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.869965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.917181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1088/2156] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:20.922607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.938500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c448bf4-ae07-40c7-8c5b-d489d43897d9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.008506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:21.035599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.036282 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.036898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.037200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.037889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.040554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.075784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1069/2157] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:21.084049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.112461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bec2e5-5bbd-4284-9146-f27c3d19160b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.125896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.127336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.127844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.132593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.132593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.133569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.159562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1089/2158] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:21.161004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.184035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba228d6-3db0-4f4e-8fa5-5155c53d309b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.205592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b791c2c-2926-489b-93aa-30a41d162532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.206379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b791c2c-2926-489b-93aa-30a41d162532 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.206664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b791c2c-2926-489b-93aa-30a41d162532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.207241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b791c2c-2926-489b-93aa-30a41d162532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.208171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b791c2c-2926-489b-93aa-30a41d162532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.216281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b791c2c-2926-489b-93aa-30a41d162532 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.239122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1070/2159] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:21.244491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.257294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739459e-098a-42eb-bc46-7abd43e95208 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.263517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.264284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.264439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.264688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] PATH_INFO: `/v3/projects/d5c8c5bf1b1c4caeab8b5daf28ada57d/users/c4284eb173fc49d1b059fdd2f3de2b4d/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.284653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c8c5bf1b1c4caeab8b5daf28ada57d, user_id=c4284eb173fc49d1b059fdd2f3de2b4d, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.287509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.310882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1090/2160] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:21 2022] PUT /identity/v3/projects/d5c8c5bf1b1c4caeab8b5daf28ada57d/users/c4284eb173fc49d1b059fdd2f3de2b4d/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:21.318728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.319045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.319476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.352957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:21.354527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:21.445245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1071/2161] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:21.458380 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.462612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:21.561084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948cd76-07ad-438f-a190-7932bc323465 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.567050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302d1d7-beab-46a9-93fa-1527e847bfe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.567917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302d1d7-beab-46a9-93fa-1527e847bfe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.568513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302d1d7-beab-46a9-93fa-1527e847bfe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.582342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302d1d7-beab-46a9-93fa-1527e847bfe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.588481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302d1d7-beab-46a9-93fa-1527e847bfe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.592368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302d1d7-beab-46a9-93fa-1527e847bfe1 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.628572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1091/2162] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:25.753954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.757285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:25.791185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2c653-53e9-4848-b532-a3d69f584b91 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.800956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.802732 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.803387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.804231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] PATH_INFO: `/v3/users/3d5ed6131771464d9557dd595da1018a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.808055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5ed6131771464d9557dd595da1018a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.815204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.952056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5ed6131771464d9557dd595da1018a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.982500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5ed6131771464d9557dd595da1018a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.985078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5ed6131771464d9557dd595da1018a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.012991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5ed6131771464d9557dd595da1018a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.058687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5ed6131771464d9557dd595da1018a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.081771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.083082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.083731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.090907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1072/2163] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:25 2022] DELETE /identity/v3/users/3d5ed6131771464d9557dd595da1018a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:26.131559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:26.133984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:26.153502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.232301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1092/2164] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:26.246903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.248671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:26.312836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.342894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb0176-6582-45d7-876c-89f6b2e8f7bd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.349187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109554d-79d5-4ebe-aa68-c19122097799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.350780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109554d-79d5-4ebe-aa68-c19122097799 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.351556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109554d-79d5-4ebe-aa68-c19122097799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.351931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109554d-79d5-4ebe-aa68-c19122097799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.366747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109554d-79d5-4ebe-aa68-c19122097799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.368951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109554d-79d5-4ebe-aa68-c19122097799 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.381577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1073/2165] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:26.959314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.963912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:26.993031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b724e09-cc57-4372-af2a-6306b56f4a84 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:27.003892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:27.005571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:27.006307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:27.006947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] PATH_INFO: `/v3/projects/b52981ec207b4fd2b29b2a7cad7af0f6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:27.011950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52981ec207b4fd2b29b2a7cad7af0f6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:27.016332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:27.173920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52981ec207b4fd2b29b2a7cad7af0f6', 'request_id': 'req-6efe1273-84b1-498d-ba2f-93d4dfc061ec'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.202836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52981ec207b4fd2b29b2a7cad7af0f6', 'request_id': 'req-6efe1273-84b1-498d-ba2f-93d4dfc061ec'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.232598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52981ec207b4fd2b29b2a7cad7af0f6', 'request_id': 'req-6efe1273-84b1-498d-ba2f-93d4dfc061ec'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.278248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] The token cache is being invalidate because project b52981ec207b4fd2b29b2a7cad7af0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:27.278824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.280079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.283547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1093/2166] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:26 2022] DELETE /identity/v3/projects/b52981ec207b4fd2b29b2a7cad7af0f6 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:28.402482 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.403309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.403832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.453419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] MFA Rules not processed for user `c4284eb173fc49d1b059fdd2f3de2b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:28.455724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.567095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1074/2167] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:28.578092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.591307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.752527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe1273-84b1-498d-ba2f-93d4dfc061ec admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.761401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c9734-4d79-465d-86b6-bcf4b9a55021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.762223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c9734-4d79-465d-86b6-bcf4b9a55021 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.762604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c9734-4d79-465d-86b6-bcf4b9a55021 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.763037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c9734-4d79-465d-86b6-bcf4b9a55021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.778430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c9734-4d79-465d-86b6-bcf4b9a55021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.780218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c9734-4d79-465d-86b6-bcf4b9a55021 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.794084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1094/2168] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:29.356597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.361091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:29.465405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976beb85-dc9d-4534-8108-0b8d48193236 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.472080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.473005 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.473292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.473570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] PATH_INFO: `/v3/users/47f29ba2bed04cf5b79bc45a99fb567a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.475111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f29ba2bed04cf5b79bc45a99fb567a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.478801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.579864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f29ba2bed04cf5b79bc45a99fb567a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.596182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f29ba2bed04cf5b79bc45a99fb567a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.615742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f29ba2bed04cf5b79bc45a99fb567a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.662559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f29ba2bed04cf5b79bc45a99fb567a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.701749 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.726765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f29ba2bed04cf5b79bc45a99fb567a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.738162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1075/2169] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:29 2022] DELETE /identity/v3/users/47f29ba2bed04cf5b79bc45a99fb567a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:29.749834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.753451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.754183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.794735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:29.797641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:29.856102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.003143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1095/2170] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:30.021488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.023830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.225825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae2ed-5048-4517-be1f-b2beb3f5e196 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.239452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.240229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.240521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.240812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.282081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.287035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.317043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1076/2171] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:31.026943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.030315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:31.058991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435a43e-d4d0-4894-b2af-166cdb1e7cd1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.074275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.077465 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.078508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.079691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] PATH_INFO: `/v3/projects/103f3495c3844c8db2f60543bbcb73a1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.083721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=103f3495c3844c8db2f60543bbcb73a1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.100833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.222345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '103f3495c3844c8db2f60543bbcb73a1', 'request_id': 'req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.239009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '103f3495c3844c8db2f60543bbcb73a1', 'request_id': 'req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.251947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '103f3495c3844c8db2f60543bbcb73a1', 'request_id': 'req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.282477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] The token cache is being invalidate because project 103f3495c3844c8db2f60543bbcb73a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:31.283022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.284207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cf6238-3e3a-4c73-b9e1-83df0a8ab488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.290533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1096/2172] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:31 2022] DELETE /identity/v3/projects/103f3495c3844c8db2f60543bbcb73a1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:31.292653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.316762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd79023-1f09-414e-b28e-2faa9ee34d2b service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.551018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.552149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.552500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.553162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] PATH_INFO: `/v3/users/99f9e1e1e9ad4503874321aef571dad8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.554618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f9e1e1e9ad4503874321aef571dad8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.558501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.683680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f9e1e1e9ad4503874321aef571dad8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.715702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f9e1e1e9ad4503874321aef571dad8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.716265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f9e1e1e9ad4503874321aef571dad8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.746547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f9e1e1e9ad4503874321aef571dad8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.774336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f9e1e1e9ad4503874321aef571dad8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.815619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1077/2173] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:31 2022] DELETE /identity/v3/users/99f9e1e1e9ad4503874321aef571dad8 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:31.913738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a67bc3-8609-475a-848f-7b09b687faf5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.914189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a67bc3-8609-475a-848f-7b09b687faf5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.915187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a67bc3-8609-475a-848f-7b09b687faf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.015028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-30a67bc3-8609-475a-848f-7b09b687faf5 None None] MFA Rules not processed for user `c4284eb173fc49d1b059fdd2f3de2b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:32.018360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-30a67bc3-8609-475a-848f-7b09b687faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:32.127941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1097/2174] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:33.252974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.257859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:33.288016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:33.495315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd9f8f-1aa3-44fd-9a49-83fd20a0271f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.522177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.523140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.523420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.525184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] PATH_INFO: `/v3/projects/107186a562dc4472a981b9e36393ed06` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.543546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=107186a562dc4472a981b9e36393ed06)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:33.567680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:33.838487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107186a562dc4472a981b9e36393ed06', 'request_id': 'req-dfdc4563-c232-459e-81bb-080f355339f6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.869100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107186a562dc4472a981b9e36393ed06', 'request_id': 'req-dfdc4563-c232-459e-81bb-080f355339f6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.890538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107186a562dc4472a981b9e36393ed06', 'request_id': 'req-dfdc4563-c232-459e-81bb-080f355339f6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.919102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] The token cache is being invalidate because project 107186a562dc4472a981b9e36393ed06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:33.920576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.923688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-dfdc4563-c232-459e-81bb-080f355339f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.931689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1078/2175] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:33 2022] DELETE /identity/v3/projects/107186a562dc4472a981b9e36393ed06 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:33.941170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.941770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.941770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.978260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:33.979918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.129215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.189467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1098/2176] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:34.211958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.214839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.215519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.295390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:34.299404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.326019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1079/2177] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:34.331467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.350209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763ba33-3151-4bfa-a668-94010d0c71fb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.367076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.368194 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.368652 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.369110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.370582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:34.371253 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.383078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.403340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1099/2178] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:34.408909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.428891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594aa9b-b397-4772-8a51-e371db37051e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.436846 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.437878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.438374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.438838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.440031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.442421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.483322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1080/2179] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:34.491494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.517290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035a0f-14c4-4387-a86a-7eaf2ae4e670 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.707963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.708827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.709203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.709484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.710467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.714004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.748172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1100/2180] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:34.754487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.771019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac03c-bb62-488c-a12e-9374f0d9d91a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.778417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.779140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.779631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.780551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.781302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.783236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.797457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1081/2181] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:34.802192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.815418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb502cf-3126-4c17-a17d-e4bb6d51aba7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.821175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5dec3-d15f-4f74-92d9-6ed85b7ae67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.822026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5dec3-d15f-4f74-92d9-6ed85b7ae67a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.823359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5dec3-d15f-4f74-92d9-6ed85b7ae67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.823745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5dec3-d15f-4f74-92d9-6ed85b7ae67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.824608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5dec3-d15f-4f74-92d9-6ed85b7ae67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.826520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5dec3-d15f-4f74-92d9-6ed85b7ae67a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.843937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1101/2182] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:34.845693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.861225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8de0c7-c7f8-4b58-9dc7-f3c0ce9cfdd7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.871213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.872607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.873224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.873754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] PATH_INFO: `/v3/projects/321ecc8a517b45d9b7b86acc71c54b50/users/9bdd7f3261e64e39a711a9bf037f38b6/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.903179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321ecc8a517b45d9b7b86acc71c54b50, user_id=9bdd7f3261e64e39a711a9bf037f38b6, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.906932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.925343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1082/2183] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:34 2022] PUT /identity/v3/projects/321ecc8a517b45d9b7b86acc71c54b50/users/9bdd7f3261e64e39a711a9bf037f38b6/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:34.943038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.943842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.944764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.991733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] MFA Rules not processed for user `9bdd7f3261e64e39a711a9bf037f38b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:34.995388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.140892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1102/2184] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:35.146538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.215196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.253042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5132640-0e7c-4000-b04e-7759b0d78344 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.259346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.260260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.260625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.260912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.261686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.263760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.306726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1083/2185] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:35 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:35.308066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.321897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7fa6c-d823-4822-ae07-8ff8349bce76 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.401276 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.422558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.423506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.423855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.424226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.425491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.428706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.460867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1103/2186] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:35 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:35.466308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.479694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc53d9-8fba-454a-baa6-918c196c88fe admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.485917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.486740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.487223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.487664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.488696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.490492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.509727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1084/2187] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:35.514585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.532766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e0034-69bc-419d-af7e-6d376a3fd76f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.542666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.544129 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.544563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.544995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] PATH_INFO: `/v3/projects/105ad37f36ba4a498d32d2c8eb159267/users/f70a57f3be22478e95a8ce62a114a7c8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.575631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=105ad37f36ba4a498d32d2c8eb159267, user_id=f70a57f3be22478e95a8ce62a114a7c8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.578963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.599863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1104/2188] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:35 2022] PUT /identity/v3/projects/105ad37f36ba4a498d32d2c8eb159267/users/f70a57f3be22478e95a8ce62a114a7c8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:35.610931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.624895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499870b-c45d-48b2-ad70-6d6931baa6b4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.727682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd3163-f837-4277-a1fb-7397eeca3501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.728729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd3163-f837-4277-a1fb-7397eeca3501 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.729015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd3163-f837-4277-a1fb-7397eeca3501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.729291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd3163-f837-4277-a1fb-7397eeca3501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.730060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd3163-f837-4277-a1fb-7397eeca3501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.731905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd3163-f837-4277-a1fb-7397eeca3501 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.750523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1085/2189] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:35.751809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.770147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181712-6560-4727-b4e8-d594ea2eee1c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.778761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.780348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.780827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.781301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] PATH_INFO: `/v3/projects/105ad37f36ba4a498d32d2c8eb159267/users/f70a57f3be22478e95a8ce62a114a7c8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.784793 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=105ad37f36ba4a498d32d2c8eb159267, user_id=f70a57f3be22478e95a8ce62a114a7c8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.796967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.829216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1105/2190] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:35 2022] PUT /identity/v3/projects/105ad37f36ba4a498d32d2c8eb159267/users/f70a57f3be22478e95a8ce62a114a7c8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:35.839151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.840843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.842046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.891393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] MFA Rules not processed for user `f70a57f3be22478e95a8ce62a114a7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:35.892892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.994038 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1086/2191] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:36.005708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.007475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.081115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477464e-555c-4cb3-98b8-174bc5401228 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.085237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ee76c-3ec9-4426-be47-9c6cab96071f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.085917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ee76c-3ec9-4426-be47-9c6cab96071f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.086203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ee76c-3ec9-4426-be47-9c6cab96071f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.086648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ee76c-3ec9-4426-be47-9c6cab96071f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.155046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ee76c-3ec9-4426-be47-9c6cab96071f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.156794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ee76c-3ec9-4426-be47-9c6cab96071f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.171854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1106/2192] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:40.155298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.214832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.287606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db71ca-f8ca-4669-a58b-b4e99b33c3f3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.302587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.304889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.305943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.306573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] PATH_INFO: `/v3/users/9bdd7f3261e64e39a711a9bf037f38b6` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.322179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bdd7f3261e64e39a711a9bf037f38b6)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.329568 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.494055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bdd7f3261e64e39a711a9bf037f38b6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:40.525756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bdd7f3261e64e39a711a9bf037f38b6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:40.526670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bdd7f3261e64e39a711a9bf037f38b6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:40.552146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bdd7f3261e64e39a711a9bf037f38b6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:40.609759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bdd7f3261e64e39a711a9bf037f38b6'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:40.630476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.631387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.631387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.647101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1087/2193] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:40 2022] DELETE /identity/v3/users/9bdd7f3261e64e39a711a9bf037f38b6 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:40.696761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:40.714335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:40.864512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.907769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1107/2194] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:40.919805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.922938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.086686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6825c7-8d0b-40c8-bf79-dbf09f0c54de admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.095375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.096228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.096664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.097172 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.112836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.114665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.143649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1088/2195] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:41.635646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.637896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.669544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190974-082b-491d-88a8-08fe31a27eb3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.677036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.678324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.678788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.679358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] PATH_INFO: `/v3/projects/321ecc8a517b45d9b7b86acc71c54b50` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.682382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321ecc8a517b45d9b7b86acc71c54b50)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.685931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.763330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321ecc8a517b45d9b7b86acc71c54b50', 'request_id': 'req-bc691689-15da-46b6-af47-59614a61e137'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.772708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321ecc8a517b45d9b7b86acc71c54b50', 'request_id': 'req-bc691689-15da-46b6-af47-59614a61e137'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.782482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321ecc8a517b45d9b7b86acc71c54b50', 'request_id': 'req-bc691689-15da-46b6-af47-59614a61e137'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.800856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] The token cache is being invalidate because project 321ecc8a517b45d9b7b86acc71c54b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:41.801169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.801947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.803835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1108/2196] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:41 2022] DELETE /identity/v3/projects/321ecc8a517b45d9b7b86acc71c54b50 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:41.809842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.826591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc1874-62ab-4f33-a377-eba9b6379cf3 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.880338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.892456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.893551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.894358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.899526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] PATH_INFO: `/v3/users/f70a57f3be22478e95a8ce62a114a7c8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.901527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f70a57f3be22478e95a8ce62a114a7c8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.905591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.979028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f70a57f3be22478e95a8ce62a114a7c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.989418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f70a57f3be22478e95a8ce62a114a7c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.990336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f70a57f3be22478e95a8ce62a114a7c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.003697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f70a57f3be22478e95a8ce62a114a7c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.029693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f9494e3-d2d5-42e4-b60d-b4d89897d67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f70a57f3be22478e95a8ce62a114a7c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.037248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1089/2197] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:41 2022] DELETE /identity/v3/users/f70a57f3be22478e95a8ce62a114a7c8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:42.431873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.434675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:42.462020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691689-15da-46b6-af47-59614a61e137 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.581242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.582803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.583404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.583940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] PATH_INFO: `/v3/projects/105ad37f36ba4a498d32d2c8eb159267` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.586273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=105ad37f36ba4a498d32d2c8eb159267)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.590166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.659478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105ad37f36ba4a498d32d2c8eb159267', 'request_id': 'req-37c37b9a-ee4a-402c-9637-9280b42c2c7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.667356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105ad37f36ba4a498d32d2c8eb159267', 'request_id': 'req-37c37b9a-ee4a-402c-9637-9280b42c2c7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.674208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105ad37f36ba4a498d32d2c8eb159267', 'request_id': 'req-37c37b9a-ee4a-402c-9637-9280b42c2c7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.689825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] The token cache is being invalidate because project 105ad37f36ba4a498d32d2c8eb159267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:42.690217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.691117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-37c37b9a-ee4a-402c-9637-9280b42c2c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:42.692765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1109/2198] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:42 2022] DELETE /identity/v3/projects/105ad37f36ba4a498d32d2c8eb159267 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:42.700205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.700802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.701311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.723865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:42.725279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:42.853366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1090/2199] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:42.869523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.870673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.871263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.893482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:42.906054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:42.910524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:42.934403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1110/2200] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:42.940610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.952390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea796d7d-c7a7-4814-a6e3-9e02c124b6c9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.958134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.958831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.959185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.959679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.960440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:42.960791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.963678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.979893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1091/2201] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:42.987702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.006250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0574-8e5c-4079-8571-669de0040786 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.015341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.016369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.016697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.017178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.018507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.022756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.071828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1111/2202] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:43.076540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.088617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7156bd9-53e1-4906-b5fe-f6a428434ac8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.176115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.176808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.177092 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.177370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.178077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.180640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.188549 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.217229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1092/2203] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:43 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:43.223963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.245908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145190da-cd52-4bfc-9a38-34705b43b8c7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.251929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.252735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.253075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.253417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.254246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.256290 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.269567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1112/2204] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:43.274123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.287767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f42c-0c81-4fe5-befa-ca5a94f8e97e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.290904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.292151 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.292617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.293096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.294328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.297186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.323402 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1093/2205] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:43.327739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.349001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbf3f6-14a6-450d-8a52-3a73eae9507b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.357651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707639-46ea-4aa1-abbc-6590c66397bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.359215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707639-46ea-4aa1-abbc-6590c66397bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.359799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707639-46ea-4aa1-abbc-6590c66397bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.360328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707639-46ea-4aa1-abbc-6590c66397bf admin admin] PATH_INFO: `/v3/projects/2e2293d2cd3d4664bf4faf77c8c50995/users/71825c27304f49909bc21c8662ff1497/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.395877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707639-46ea-4aa1-abbc-6590c66397bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2293d2cd3d4664bf4faf77c8c50995, user_id=71825c27304f49909bc21c8662ff1497, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.401514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707639-46ea-4aa1-abbc-6590c66397bf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.412589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.415953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.434099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1113/2206] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:43 2022] PUT /identity/v3/projects/2e2293d2cd3d4664bf4faf77c8c50995/users/71825c27304f49909bc21c8662ff1497/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:43.439149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b9c6-14e5-4168-8276-4c3953fc4fad admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.447412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.447870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.448329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.497346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] MFA Rules not processed for user `71825c27304f49909bc21c8662ff1497`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:43.499956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.568394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.569308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.569567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.569847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] PATH_INFO: `/v3/users/a93cdd01cfed4fccb8f6571fe12692b5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.571364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a93cdd01cfed4fccb8f6571fe12692b5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.573579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.626697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1114/2207] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:43.646880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.649537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.662447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a93cdd01cfed4fccb8f6571fe12692b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:43.676626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a93cdd01cfed4fccb8f6571fe12692b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:43.677696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a93cdd01cfed4fccb8f6571fe12692b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:43.691824 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a93cdd01cfed4fccb8f6571fe12692b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:43.728339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3e5ca11e-40f7-4e67-a285-c134acbf11b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a93cdd01cfed4fccb8f6571fe12692b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:43.737699 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1094/2208] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:43 2022] DELETE /identity/v3/users/a93cdd01cfed4fccb8f6571fe12692b5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:43.740625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611c8a8-0cd6-47b9-bf4d-336d1f9280b0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.744723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.745012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.745280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.767392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:43.768711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.826933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.836216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.838344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.839285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.839856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.844606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.849343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.875460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1095/2209] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:43.876551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.878349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.957078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.959942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.973113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fda2f5-bcef-4e59-9054-091adecd5471 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.976125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1115/2210] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:43.978592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.979938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.980490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.981080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.999266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.008750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.030167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1096/2211] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:44.452892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:44.455399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:44.478986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a6afd-4c11-4dc8-b447-8c843fc8bdf6 service glance] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.485843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.487079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.487477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.487870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] PATH_INFO: `/v3/projects/001b3b31ec1d40f0b2d50f82d1267f6b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.491506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001b3b31ec1d40f0b2d50f82d1267f6b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.496804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.552203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001b3b31ec1d40f0b2d50f82d1267f6b', 'request_id': 'req-56c0285b-dbc5-4b9c-991c-9510c230c736'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:44.560150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001b3b31ec1d40f0b2d50f82d1267f6b', 'request_id': 'req-56c0285b-dbc5-4b9c-991c-9510c230c736'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:44.567909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001b3b31ec1d40f0b2d50f82d1267f6b', 'request_id': 'req-56c0285b-dbc5-4b9c-991c-9510c230c736'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:44.589873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] The token cache is being invalidate because project 001b3b31ec1d40f0b2d50f82d1267f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:44.590180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:44.590916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-56c0285b-dbc5-4b9c-991c-9510c230c736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:44.593183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1116/2212] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:44 2022] DELETE /identity/v3/projects/001b3b31ec1d40f0b2d50f82d1267f6b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:54.022023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=27 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.588108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:59.778140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:59.779810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:59.795472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812b98-f8e1-49d1-93c3-88fd469c2d85 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:59.893943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:59.894837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.895407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.895683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] PATH_INFO: `/v3/users/4b5a5f6a431148979b9278b6dd9dcb04` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.896932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5a5f6a431148979b9278b6dd9dcb04)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:59.899711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:59.959793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5a5f6a431148979b9278b6dd9dcb04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:59.968599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5a5f6a431148979b9278b6dd9dcb04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:59.969261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5a5f6a431148979b9278b6dd9dcb04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:59.977441 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5a5f6a431148979b9278b6dd9dcb04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:59.989798 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5a5f6a431148979b9278b6dd9dcb04'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:59.996389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1097/2213] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:59 2022] DELETE /identity/v3/users/4b5a5f6a431148979b9278b6dd9dcb04 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:00.000912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.001407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.001545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.021280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:00.022185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.110263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1117/2214] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:00.121183 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.122842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.195088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922bde-a76e-414d-8693-6545e80bf12a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.199367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b34d-892e-4675-b5fb-9ae2d3394c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.200066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b34d-892e-4675-b5fb-9ae2d3394c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.200336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b34d-892e-4675-b5fb-9ae2d3394c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.200615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b34d-892e-4675-b5fb-9ae2d3394c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.214309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b34d-892e-4675-b5fb-9ae2d3394c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.216681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b34d-892e-4675-b5fb-9ae2d3394c98 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.231855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1098/2215] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:00.551563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.553159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.567942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb63d4-fcd0-4194-9365-e9eba09fb4d6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.575591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.576479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.577468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.577468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] PATH_INFO: `/v3/projects/a2632b16449645b1b031a66d2ac250be` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.578395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2632b16449645b1b031a66d2ac250be)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.581791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.637803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2632b16449645b1b031a66d2ac250be', 'request_id': 'req-631034dd-594f-4122-aaf8-5153cdc768c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:00.644800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2632b16449645b1b031a66d2ac250be', 'request_id': 'req-631034dd-594f-4122-aaf8-5153cdc768c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:00.651608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2632b16449645b1b031a66d2ac250be', 'request_id': 'req-631034dd-594f-4122-aaf8-5153cdc768c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:00.672232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] The token cache is being invalidate because project a2632b16449645b1b031a66d2ac250be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:00.672560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:00.673353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-631034dd-594f-4122-aaf8-5153cdc768c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:00.675002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1118/2216] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:00 2022] DELETE /identity/v3/projects/a2632b16449645b1b031a66d2ac250be => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:00.680705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.680998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.681281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.700830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:00.702107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.721632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.771895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1099/2217] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:00.780694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.781013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.781313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.801963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:00.803235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.812421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1119/2218] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:00.816573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.827114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de26861-fa24-4250-a1fd-72b2be4b8271 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.831852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.832527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.832806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.833084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.833818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:00.834126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.836776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.848849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1100/2219] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:00.853728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.863441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb43d79-55e8-4d0e-98d3-9f4fcf4cad7f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.868656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.869336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.869621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.869897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.870611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.872594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.911737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1120/2220] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:00 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:00.916517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.927979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c3f2e-76df-4b07-89ef-41bd824bf9be admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.996513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.997208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.997494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.997775 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.998497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.000513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.027076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1101/2221] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:00 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:01.032451 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.051264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997fb50-2555-41f0-9bcc-78c39ffc7bad admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.058301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.059507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.060015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.060518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.061915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.065098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.086890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1121/2222] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:01.091469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.121740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0270b5-1091-4194-95cc-8b0abb68f916 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.129848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e34ae4-3d29-461c-bca0-a6a60a31b0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.131006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e34ae4-3d29-461c-bca0-a6a60a31b0dc admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.131562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e34ae4-3d29-461c-bca0-a6a60a31b0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.132633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e34ae4-3d29-461c-bca0-a6a60a31b0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.133938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e34ae4-3d29-461c-bca0-a6a60a31b0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.135989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e34ae4-3d29-461c-bca0-a6a60a31b0dc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.152923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1102/2223] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:01.157858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.174605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db588a1-c973-45e3-95f5-ea65d2c1a3cf admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.183247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.184860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.185372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.185777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] PATH_INFO: `/v3/projects/7a302ecda6ca4e778fadc4d9dcbef1e2/users/13dbb228e1c846f48d37b43be84045f9/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.207506 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a302ecda6ca4e778fadc4d9dcbef1e2, user_id=13dbb228e1c846f48d37b43be84045f9, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.212108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.231752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1122/2224] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:01 2022] PUT /identity/v3/projects/7a302ecda6ca4e778fadc4d9dcbef1e2/users/13dbb228e1c846f48d37b43be84045f9/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:01.237508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c314-e60a-47b1-a213-ee0d374ac439 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.237836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c314-e60a-47b1-a213-ee0d374ac439 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.238126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c314-e60a-47b1-a213-ee0d374ac439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.259976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f3b6c314-e60a-47b1-a213-ee0d374ac439 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:01.261064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f3b6c314-e60a-47b1-a213-ee0d374ac439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.333030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1103/2225] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:01.345294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.347450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.371761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.437118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cefd8-4356-4320-8857-35f5fd977006 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.441127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.441844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.442133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.442410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.454882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.456490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.468236 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1123/2226] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:08.474904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.476944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.477469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.514704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] MFA Rules not processed for user `13dbb228e1c846f48d37b43be84045f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:08.516132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.543174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.615363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1104/2227] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:08.620289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.630807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b603d-e22e-496d-b5eb-2b6f1ea6f358 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.640327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.641018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.641305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.641585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.642314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.644341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.686611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1124/2228] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:08.691134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.703829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea5133-d691-48d6-9999-c5cc48e9937a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.768923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.769503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.770043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.770323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.771248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.774065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.800898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1105/2229] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:08.805486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.816570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7462fe-54cd-4107-aa94-6cffceff6e01 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.821483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.822333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.822700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.823042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.823856 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.825721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.838720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1125/2230] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:08.843173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.855247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9c9da-897e-4032-8a5b-581a7c9b0a01 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.860408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.861320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.861602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.861880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] PATH_INFO: `/v3/projects/14b53908046448feacc73140cee74646/users/2b934a8ee3ac47deaf7c7a3c578f45cf/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.879872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b53908046448feacc73140cee74646, user_id=2b934a8ee3ac47deaf7c7a3c578f45cf, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.883307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.902762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1106/2231] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:08 2022] PUT /identity/v3/projects/14b53908046448feacc73140cee74646/users/2b934a8ee3ac47deaf7c7a3c578f45cf/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:08.907538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.919314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc7deb-4300-4a83-b91a-eb34d7c4def1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.004791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a824f5-2bcd-4591-b75e-0e6b03d6cc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.005495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a824f5-2bcd-4591-b75e-0e6b03d6cc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.005784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a824f5-2bcd-4591-b75e-0e6b03d6cc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.006068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a824f5-2bcd-4591-b75e-0e6b03d6cc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.007369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a824f5-2bcd-4591-b75e-0e6b03d6cc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.009150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a824f5-2bcd-4591-b75e-0e6b03d6cc18 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.024693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1126/2232] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:09.030545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.049579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864013-0e60-47a1-8c81-d5678ee9c802 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.059731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.061236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.061671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.062120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] PATH_INFO: `/v3/projects/14b53908046448feacc73140cee74646/users/2b934a8ee3ac47deaf7c7a3c578f45cf/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.065351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b53908046448feacc73140cee74646, user_id=2b934a8ee3ac47deaf7c7a3c578f45cf, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.068628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.089391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1107/2233] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:09 2022] PUT /identity/v3/projects/14b53908046448feacc73140cee74646/users/2b934a8ee3ac47deaf7c7a3c578f45cf/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:16.728509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbd7d4-6ee5-4ee9-91ae-40b058b8e627 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.729058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbd7d4-6ee5-4ee9-91ae-40b058b8e627 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.729675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbd7d4-6ee5-4ee9-91ae-40b058b8e627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.788915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-30fbd7d4-6ee5-4ee9-91ae-40b058b8e627 None None] MFA Rules not processed for user `2b934a8ee3ac47deaf7c7a3c578f45cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:16.790464 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-30fbd7d4-6ee5-4ee9-91ae-40b058b8e627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.895640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1127/2234] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:16.908864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.910931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.016530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629437ab-9476-4ff3-be4b-6b8bd981b705 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.024026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.025176 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.025698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.026205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.077579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.126789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.126789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.135087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1108/2235] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:17.264548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621a4f7-7103-409a-81ac-f5bdd4709ad0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.264818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621a4f7-7103-409a-81ac-f5bdd4709ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.265096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621a4f7-7103-409a-81ac-f5bdd4709ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.285737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9621a4f7-7103-409a-81ac-f5bdd4709ad0 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:17.287005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9621a4f7-7103-409a-81ac-f5bdd4709ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.293162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.357724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1128/2236] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:35:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:17.416107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.417711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.496596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f07e84-a907-471c-9454-41c793e0e398 service glance] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.502079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.502758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.503045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.503357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.513350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.514972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.526881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1109/2237] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:35:19.465799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.467552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.468750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.540751 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:19.546334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.586909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1129/2238] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:19.629430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.706197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9f91-1d71-47dc-b241-af91cfe38dac service swift] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.730377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661d31-14cc-4787-81fa-b66d0b6074dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.732261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661d31-14cc-4787-81fa-b66d0b6074dc service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.732812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661d31-14cc-4787-81fa-b66d0b6074dc service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.733310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661d31-14cc-4787-81fa-b66d0b6074dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.759326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5661d31-14cc-4787-81fa-b66d0b6074dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.761825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5661d31-14cc-4787-81fa-b66d0b6074dc service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.793506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1110/2239] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:35:24.064259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.065767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.288839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439cc26-2f0c-45e8-b7b0-096bab025053 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.294744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.296139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.297113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.297618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] PATH_INFO: `/v3/users/71825c27304f49909bc21c8662ff1497` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.303828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71825c27304f49909bc21c8662ff1497)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.306576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.418552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71825c27304f49909bc21c8662ff1497'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.437706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71825c27304f49909bc21c8662ff1497'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.462262 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71825c27304f49909bc21c8662ff1497'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.463488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71825c27304f49909bc21c8662ff1497'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.486684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71825c27304f49909bc21c8662ff1497'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.527376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1130/2240] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:24 2022] DELETE /identity/v3/users/71825c27304f49909bc21c8662ff1497 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:24.530899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136179c-2c44-432c-b299-c90bd5651be3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.532872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136179c-2c44-432c-b299-c90bd5651be3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.533696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136179c-2c44-432c-b299-c90bd5651be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.583734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0136179c-2c44-432c-b299-c90bd5651be3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:24.587452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0136179c-2c44-432c-b299-c90bd5651be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.795033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1111/2241] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:24.840558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.842546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.919178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:24.998613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa37759-ab95-48c2-ae3c-6914a2fe2485 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.009049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.009748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.011010 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.011288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.044992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.047857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.118021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1131/2242] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:25.250479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2f3e7-9e24-4b4e-ae67-a5188df77ef9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.250891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2f3e7-9e24-4b4e-ae67-a5188df77ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.251323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2f3e7-9e24-4b4e-ae67-a5188df77ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.300484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-59c2f3e7-9e24-4b4e-ae67-a5188df77ef9 None None] MFA Rules not processed for user `cf3fd4428e3b46cd9e83e1d32571097b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:25.301995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-59c2f3e7-9e24-4b4e-ae67-a5188df77ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:25.432456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.566799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1112/2243] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:26.131673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.136258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:26.225827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5551db-3a72-4baa-a5ab-277e7ab12a0e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.278601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.281404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.282126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.282736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] PATH_INFO: `/v3/projects/2e2293d2cd3d4664bf4faf77c8c50995` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.289699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2293d2cd3d4664bf4faf77c8c50995)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.293432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.564388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2293d2cd3d4664bf4faf77c8c50995', 'request_id': 'req-cba8641e-3261-445e-aba8-6260217f9477'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:26.584665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2293d2cd3d4664bf4faf77c8c50995', 'request_id': 'req-cba8641e-3261-445e-aba8-6260217f9477'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:26.600230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2293d2cd3d4664bf4faf77c8c50995', 'request_id': 'req-cba8641e-3261-445e-aba8-6260217f9477'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:26.666291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] The token cache is being invalidate because project 2e2293d2cd3d4664bf4faf77c8c50995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:26.667045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:26.668646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cba8641e-3261-445e-aba8-6260217f9477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:26.672361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1132/2244] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:26 2022] DELETE /identity/v3/projects/2e2293d2cd3d4664bf4faf77c8c50995 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:33.311421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.312131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.312131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.356901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] MFA Rules not processed for user `13dbb228e1c846f48d37b43be84045f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:33.357924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:33.491240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1113/2245] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:36.661094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.090872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724dac2-61b9-4279-9f43-5511a7f05673 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.091456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724dac2-61b9-4279-9f43-5511a7f05673 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.091948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724dac2-61b9-4279-9f43-5511a7f05673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.118130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c724dac2-61b9-4279-9f43-5511a7f05673 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:37.119740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c724dac2-61b9-4279-9f43-5511a7f05673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:37.187663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1133/2246] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:35:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:37.202348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.203913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:37.296573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f29d7-1753-4d2a-b389-ccb4852c110a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.303145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5124a-af1e-454a-a70f-29a61f43050e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.304195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5124a-af1e-454a-a70f-29a61f43050e service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.304479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5124a-af1e-454a-a70f-29a61f43050e service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.304754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5124a-af1e-454a-a70f-29a61f43050e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.315407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5124a-af1e-454a-a70f-29a61f43050e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.318116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5124a-af1e-454a-a70f-29a61f43050e service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.330406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1114/2247] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:35:47.184195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=25 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.327895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=63 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.638280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4e48b-e0f5-474b-ac6e-3c41fb71329c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.638917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4e48b-e0f5-474b-ac6e-3c41fb71329c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.638917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4e48b-e0f5-474b-ac6e-3c41fb71329c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.640283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1134/2248] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:50.647404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19e14-f8d7-4736-831d-4fbe95eb9342 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.647969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19e14-f8d7-4736-831d-4fbe95eb9342 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.647969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19e14-f8d7-4736-831d-4fbe95eb9342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.668458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6af19e14-f8d7-4736-831d-4fbe95eb9342 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:50.669779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6af19e14-f8d7-4736-831d-4fbe95eb9342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.757724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1115/2249] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:50.776852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4e48b-e0f5-474b-ac6e-3c41fb71329c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.778640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-85a4e48b-e0f5-474b-ac6e-3c41fb71329c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.875025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4e48b-e0f5-474b-ac6e-3c41fb71329c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.879325 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d57f0-753b-4bd9-9755-b5c6b13a43d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.880157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d57f0-753b-4bd9-9755-b5c6b13a43d3 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.880519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d57f0-753b-4bd9-9755-b5c6b13a43d3 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.880869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d57f0-753b-4bd9-9755-b5c6b13a43d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.893758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d57f0-753b-4bd9-9755-b5c6b13a43d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.896410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d57f0-753b-4bd9-9755-b5c6b13a43d3 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.908344 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1135/2250] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:50.984350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f23714-39a2-4dd0-a993-409bf08c2f07 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.984644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f23714-39a2-4dd0-a993-409bf08c2f07 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.984928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f23714-39a2-4dd0-a993-409bf08c2f07 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.986326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1116/2251] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:50.991475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.991776 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.992084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.013149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] MFA Rules not processed for user `7f5dceea71d1434796a105ea67773cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:51.014439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.081256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1136/2252] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:51.093618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f23714-39a2-4dd0-a993-409bf08c2f07 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.095659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-49f23714-39a2-4dd0-a993-409bf08c2f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.110589 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f23714-39a2-4dd0-a993-409bf08c2f07 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.116954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a3e41-ed21-42ee-a5b3-8547ed00b7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.117724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a3e41-ed21-42ee-a5b3-8547ed00b7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.118073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a3e41-ed21-42ee-a5b3-8547ed00b7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.118453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a3e41-ed21-42ee-a5b3-8547ed00b7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.126907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a3e41-ed21-42ee-a5b3-8547ed00b7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.128484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a3e41-ed21-42ee-a5b3-8547ed00b7fe service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.137597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1117/2253] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:51.266888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.273211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.300166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b1b5-d945-4c45-950d-23814e5012a0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.300455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b1b5-d945-4c45-950d-23814e5012a0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.300722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b1b5-d945-4c45-950d-23814e5012a0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.303451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1118/2254] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:51.311206 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.311613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.312069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.341969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] MFA Rules not processed for user `054c95ae7aa645b7ade6390e9bf0059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:51.343524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.356730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1119/2255] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:51.370766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.372317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.372928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e199ab-7477-4d35-bda3-60bec1352486 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.378806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aad36-2928-4960-ab37-000e09524480 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.381085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aad36-2928-4960-ab37-000e09524480 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.382410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aad36-2928-4960-ab37-000e09524480 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.382851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aad36-2928-4960-ab37-000e09524480 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.385655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11b26c-ad95-438d-9714-4de9bfc3fe71 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.385983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-171aad36-2928-4960-ab37-000e09524480 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.390548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b0bc3-e9e0-4d60-9d63-ff3749d1fdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.391291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0bc3-e9e0-4d60-9d63-ff3749d1fdaa service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.391577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0bc3-e9e0-4d60-9d63-ff3749d1fdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.391855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0bc3-e9e0-4d60-9d63-ff3749d1fdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.404318 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b0bc3-e9e0-4d60-9d63-ff3749d1fdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.404948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aad36-2928-4960-ab37-000e09524480 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.405790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b0bc3-e9e0-4d60-9d63-ff3749d1fdaa service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.407533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aad36-2928-4960-ab37-000e09524480 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.423014 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1120/2256] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:51.434291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1137/2257] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:59.989598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.990202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.990509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.022562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] MFA Rules not processed for user `c4284eb173fc49d1b059fdd2f3de2b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:00.025175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.140008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1121/2258] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:35:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:00.784967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.786165 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.786794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.810269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:00.812578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.826375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1138/2259] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:36:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:00.851661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.876532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b347c0-9fba-4b3e-aac4-16ff93a2ee0a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.883032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7545b-142f-4107-a7b5-ed79ae0f9d4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.883995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7545b-142f-4107-a7b5-ed79ae0f9d4b service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.884346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7545b-142f-4107-a7b5-ed79ae0f9d4b service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.884902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7545b-142f-4107-a7b5-ed79ae0f9d4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.896660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7545b-142f-4107-a7b5-ed79ae0f9d4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.898514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7545b-142f-4107-a7b5-ed79ae0f9d4b service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.913325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1122/2260] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:36:10.804045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:10.905627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=77 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:19.277692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:19.288896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04700e9-3b25-41d9-b62b-a25f62bec6bf None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:19.293004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:19.293726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:19.294049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:19.294365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:19.296563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:19.370289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:19.371814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:19.383573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1139/2261] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:26.023140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02576b5d-2eae-4f1f-a32d-d460dd5c569a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.024165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02576b5d-2eae-4f1f-a32d-d460dd5c569a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.024633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02576b5d-2eae-4f1f-a32d-d460dd5c569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.046977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-02576b5d-2eae-4f1f-a32d-d460dd5c569a None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:26.048464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-02576b5d-2eae-4f1f-a32d-d460dd5c569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:26.069665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1123/2262] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jul 21 22:36:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:26.099820 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:26.125026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816b0b0-6553-49dd-925d-fc32d120c636 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:26.136831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc1e55-cf03-4c36-9a42-073fd716c5ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:26.138613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc1e55-cf03-4c36-9a42-073fd716c5ed service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.139445 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc1e55-cf03-4c36-9a42-073fd716c5ed service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.140220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc1e55-cf03-4c36-9a42-073fd716c5ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.161678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc1e55-cf03-4c36-9a42-073fd716c5ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:26.166077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc1e55-cf03-4c36-9a42-073fd716c5ed service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:26.199216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1140/2263] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:36:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:36:36.040532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:36.177956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=36 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:14.214967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba6e03-2f3e-4bc8-abfe-45c5ac9c5e60 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:14.215815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba6e03-2f3e-4bc8-abfe-45c5ac9c5e60 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:14.215815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba6e03-2f3e-4bc8-abfe-45c5ac9c5e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:14.235084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a5ba6e03-2f3e-4bc8-abfe-45c5ac9c5e60 None None] MFA Rules not processed for user `cf3fd4428e3b46cd9e83e1d32571097b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:14.236924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5ba6e03-2f3e-4bc8-abfe-45c5ac9c5e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:14.307012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1124/2264] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:37:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:24.294960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=25 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:29.075209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ffb27-0fcd-4dcf-934c-92042020c14e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:29.075809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ffb27-0fcd-4dcf-934c-92042020c14e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:29.075918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ffb27-0fcd-4dcf-934c-92042020c14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:29.106592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2c0ffb27-0fcd-4dcf-934c-92042020c14e None None] MFA Rules not processed for user `a6146c8083d04171863a4b603fd6a46c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:29.108164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2c0ffb27-0fcd-4dcf-934c-92042020c14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:29.183998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1141/2265] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:30.721126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:30.721723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:30.721881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:30.740317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] MFA Rules not processed for user `c4284eb173fc49d1b059fdd2f3de2b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:30.741522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:30.749269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1125/2266] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:37:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:37:39.176259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=25 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:40.401704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:40.402526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:40.402526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:40.420953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] MFA Rules not processed for user `1ee22783b3234c4f905fedce7d59f01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:40.422525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:40.495993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1142/2267] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:37:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:40.734227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:47.602719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:47.604261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:47.723471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:47.739905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05748-8776-4a30-bd17-fadd39257420 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:47.740118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dd5fd-d00d-4248-8560-229a46f7a45f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:47.744795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:47.746035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:47.746688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:47.747319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:47.749683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:47.750356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:47.750646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:47.750930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:47.753238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:47.755641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:47.773419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:47.775363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b270c8-d7bd-41c5-8607-29651b1042a9 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:47.791928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1126/2268] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:37:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:37:47.796322 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:47.798970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:47.814450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1143/2269] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:37:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:37:55.518848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:55.519512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:55.519512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:55.539074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:55.540407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:55.547784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1127/2270] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jul 21 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:55.607271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:55.619941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da98cfc-9d0a-4a7a-a818-ce14568ea70f service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:55.626662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e900-fc86-4e6d-8485-fb15affc9a35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:55.628011 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e900-fc86-4e6d-8485-fb15affc9a35 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:55.628599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e900-fc86-4e6d-8485-fb15affc9a35 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:55.628599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e900-fc86-4e6d-8485-fb15affc9a35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:55.637618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e900-fc86-4e6d-8485-fb15affc9a35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:55.639272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e900-fc86-4e6d-8485-fb15affc9a35 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:55.648627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1144/2271] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:37:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:38:01.793510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:01.804967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970951c-daeb-4bc6-83cd-5222249dcc1e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:01.808917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:01.809595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:01.809882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:01.810344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:01.812679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:01.831645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:01.834533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c898067-41ed-44a4-8a07-176a997a4589 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:01.864512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1128/2272] 10.222.0.58 () {60 vars in 1396 bytes} [Thu Jul 21 22:38:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:38:04.596020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f551b8-176a-4a1d-838a-1e6f6a40cac1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:04.597121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f551b8-176a-4a1d-838a-1e6f6a40cac1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:04.597472 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f551b8-176a-4a1d-838a-1e6f6a40cac1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:04.599180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1145/2273] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jul 21 22:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:04.615773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:04.616380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:04.616380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:04.637149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] MFA Rules not processed for user `019c423e880647139535f2d0dfc5a9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:04.638716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:04.650334 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1129/2274] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jul 21 22:38:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:04.708650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f551b8-176a-4a1d-838a-1e6f6a40cac1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:04.726183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f551b8-176a-4a1d-838a-1e6f6a40cac1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:04.732434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:04.733128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:04.733635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:04.733748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:04.744628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:04.746287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:04.764623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1146/2275] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:38:04.775668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:04.777127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:04.860714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f979cd-7d0d-4af5-942d-52700308e65f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:04.866867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:04.867730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:04.868017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:04.868297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] PATH_INFO: `/v3/users/13dbb228e1c846f48d37b43be84045f9` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:04.869537 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=13dbb228e1c846f48d37b43be84045f9)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:04.871551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:04.942866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13dbb228e1c846f48d37b43be84045f9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:04.951737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13dbb228e1c846f48d37b43be84045f9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:04.952626 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13dbb228e1c846f48d37b43be84045f9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:04.963811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13dbb228e1c846f48d37b43be84045f9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:04.977572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:04.979222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13dbb228e1c846f48d37b43be84045f9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:04.988813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1130/2276] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:04 2022] DELETE /identity/v3/users/13dbb228e1c846f48d37b43be84045f9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:05.191860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:05.206471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61b2c-a4c3-4867-bacd-18efcb89c84f service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:05.319255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:05.321304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:05.322038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:05.322460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] PATH_INFO: `/v3/projects/7a302ecda6ca4e778fadc4d9dcbef1e2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:05.326894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a302ecda6ca4e778fadc4d9dcbef1e2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:05.335883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:05.438308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a302ecda6ca4e778fadc4d9dcbef1e2', 'request_id': 'req-543e5779-e718-40ca-b692-c259f321c982'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.447324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a302ecda6ca4e778fadc4d9dcbef1e2', 'request_id': 'req-543e5779-e718-40ca-b692-c259f321c982'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.463206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a302ecda6ca4e778fadc4d9dcbef1e2', 'request_id': 'req-543e5779-e718-40ca-b692-c259f321c982'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.468407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:05.484685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] The token cache is being invalidate because project 7a302ecda6ca4e778fadc4d9dcbef1e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:05.484878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.485593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.488803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1147/2277] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:05 2022] DELETE /identity/v3/projects/7a302ecda6ca4e778fadc4d9dcbef1e2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:05.492995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:05.494789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:05.511001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d226-e7ac-4c4b-9027-6ff5a4f4f6bb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:05.583645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:05.584277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:05.584555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:05.584978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] PATH_INFO: `/v3/users/2b934a8ee3ac47deaf7c7a3c578f45cf` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:05.586138 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b934a8ee3ac47deaf7c7a3c578f45cf)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:05.588098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:05.658711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b934a8ee3ac47deaf7c7a3c578f45cf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.668965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b934a8ee3ac47deaf7c7a3c578f45cf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.669676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b934a8ee3ac47deaf7c7a3c578f45cf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.679525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b934a8ee3ac47deaf7c7a3c578f45cf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.703712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b934a8ee3ac47deaf7c7a3c578f45cf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.713815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1131/2278] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:05 2022] DELETE /identity/v3/users/2b934a8ee3ac47deaf7c7a3c578f45cf => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:05.996074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:05.999000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:06.027413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5779-e718-40ca-b692-c259f321c982 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.120778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.121686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.121975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.122260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] PATH_INFO: `/v3/projects/14b53908046448feacc73140cee74646` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.123803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b53908046448feacc73140cee74646)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.126006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.184330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b53908046448feacc73140cee74646', 'request_id': 'req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.192920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b53908046448feacc73140cee74646', 'request_id': 'req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.199854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b53908046448feacc73140cee74646', 'request_id': 'req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.213523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] The token cache is being invalidate because project 14b53908046448feacc73140cee74646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:06.213817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.214528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.216377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1148/2279] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:05 2022] DELETE /identity/v3/projects/14b53908046448feacc73140cee74646 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:15.709320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:16.210317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:19.819103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:19.820715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:19.882950 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11346c-06df-4767-a281-b6dadfcd4c42 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:19.947180 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:19.947763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:19.948049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:19.948326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:19.950313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:20.026326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:20.027869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:20.040499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1132/2280] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:23.773197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:23.775784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:23.804250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3376f-7282-4f3a-9ac2-2b38c0835dea admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:23.812500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:23.814501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:23.815028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:23.815606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] PATH_INFO: `/v3/users/cf3fd4428e3b46cd9e83e1d32571097b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:23.817930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3fd4428e3b46cd9e83e1d32571097b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:23.821797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:23.895481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3fd4428e3b46cd9e83e1d32571097b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:23.913072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3fd4428e3b46cd9e83e1d32571097b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:23.934072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3fd4428e3b46cd9e83e1d32571097b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:23.935293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3fd4428e3b46cd9e83e1d32571097b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:23.950140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2391bcca-b973-44a2-8e94-ed08c217d734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3fd4428e3b46cd9e83e1d32571097b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:23.962436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1149/2281] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:23 2022] DELETE /identity/v3/users/cf3fd4428e3b46cd9e83e1d32571097b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:24.175505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.187500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b667b-9af1-4f2c-ba1e-3e2c984629cf service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.278953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.280505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.281004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.281504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] PATH_INFO: `/v3/projects/3138cf3bb8a24ba19295565fca0a471e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.284687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3138cf3bb8a24ba19295565fca0a471e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.288444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.303425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:24.352035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3138cf3bb8a24ba19295565fca0a471e', 'request_id': 'req-886e3f67-d4a7-4218-9bf6-b1aa851abc01'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:24.362301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3138cf3bb8a24ba19295565fca0a471e', 'request_id': 'req-886e3f67-d4a7-4218-9bf6-b1aa851abc01'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:24.370746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3138cf3bb8a24ba19295565fca0a471e', 'request_id': 'req-886e3f67-d4a7-4218-9bf6-b1aa851abc01'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:24.383638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] The token cache is being invalidate because project 3138cf3bb8a24ba19295565fca0a471e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:24.383939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:24.384644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-886e3f67-d4a7-4218-9bf6-b1aa851abc01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:24.386241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1133/2282] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:24 2022] DELETE /identity/v3/projects/3138cf3bb8a24ba19295565fca0a471e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:24.393972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.394258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.394522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.414618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:24.415959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:24.488724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1150/2283] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:24.496563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.496854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.497137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.516594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:24.518067 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:24.527700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1134/2284] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:24.532864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.543043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07978e7-349c-4072-acf1-d2c2b287c1d4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.548359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.549043 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.549323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.549631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.550360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:24.550671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.552333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.565816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1151/2285] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:38:24.570885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.581380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda78348-47bd-4ff6-876d-8628da5c14ba None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.586301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.586995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.587315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.587595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.588327 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.590290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.621376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1135/2286] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:24 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:24.626667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.638574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77980cbc-15cc-4cdd-b848-f3a641ae6536 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.710488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.711092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.711390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.711677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.712430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.714424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.722459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:24.742784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1152/2287] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:24 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:24.747205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.759023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991a5d9-4383-4cd3-9cfb-de3544d8eacb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.764572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.765270 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.765552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.765829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.767889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.769620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.781831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1136/2288] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:24.786868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.797713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9d3fe-cd1f-4661-8f7d-be139586055e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.802704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec883edf-8e0f-434a-a223-54d3de062213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.803441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec883edf-8e0f-434a-a223-54d3de062213 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.803731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec883edf-8e0f-434a-a223-54d3de062213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.804021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec883edf-8e0f-434a-a223-54d3de062213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.804778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec883edf-8e0f-434a-a223-54d3de062213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.806470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec883edf-8e0f-434a-a223-54d3de062213 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.817853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1153/2289] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:24.822428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.834016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f8b82-093f-4202-b9ab-ba4ad4613202 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.839222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.840144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.840436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.840712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] PATH_INFO: `/v3/projects/75436067372a4bf59034c620b1e4f4ba/users/52cbd0ce79d04710873f8d39e8616b35/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.859013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75436067372a4bf59034c620b1e4f4ba, user_id=52cbd0ce79d04710873f8d39e8616b35, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.862584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.879104 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1137/2290] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:24 2022] PUT /identity/v3/projects/75436067372a4bf59034c620b1e4f4ba/users/52cbd0ce79d04710873f8d39e8616b35/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:24.890522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.890812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.892027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.924490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] MFA Rules not processed for user `52cbd0ce79d04710873f8d39e8616b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:24.926663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:25.007177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1154/2291] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:25.012300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.085499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f651a-217e-4cd1-a9fc-20e8370eca22 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.090926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.091638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.091919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.092194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.092894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.094963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.123435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1138/2292] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:25 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:25.128312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.140144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8621df-8935-4454-aeb7-1bd375a2b244 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.203438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.204808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.205294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.205581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.206310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.208353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.231328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1155/2293] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:25 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:25.237278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.249341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cf821-8ea9-4561-9d3d-a3e6a89c1db4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.254871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.255594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.255881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.256159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.256964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.258663 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.271337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1139/2294] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:25.275750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.289563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9ac2-5099-4f31-8181-490e7a46411a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.294523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.295469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.295752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.296028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] PATH_INFO: `/v3/projects/7d5d6d15ce5d4c7692ba9dc3d4a262c5/users/eab68e3778bf4d1aa224cc9f46a36ebb/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.312504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5d6d15ce5d4c7692ba9dc3d4a262c5, user_id=eab68e3778bf4d1aa224cc9f46a36ebb, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.315886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.331971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1156/2295] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:25 2022] PUT /identity/v3/projects/7d5d6d15ce5d4c7692ba9dc3d4a262c5/users/eab68e3778bf4d1aa224cc9f46a36ebb/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:25.336093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.346717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff8fc7-3991-4b20-b8c0-17250628a93c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.350585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:25.411507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c7392-0104-409c-be85-9f831ffc21c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.412190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c7392-0104-409c-be85-9f831ffc21c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.412476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c7392-0104-409c-be85-9f831ffc21c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.412765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c7392-0104-409c-be85-9f831ffc21c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.413519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c7392-0104-409c-be85-9f831ffc21c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.415213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c7392-0104-409c-be85-9f831ffc21c3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.426729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1140/2296] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:25.430947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.441726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951b995-2798-4310-a9d0-904232dba448 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.446458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.447403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.447691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.447968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] PATH_INFO: `/v3/projects/7d5d6d15ce5d4c7692ba9dc3d4a262c5/users/eab68e3778bf4d1aa224cc9f46a36ebb/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.449995 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5d6d15ce5d4c7692ba9dc3d4a262c5, user_id=eab68e3778bf4d1aa224cc9f46a36ebb, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.452708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.465056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1157/2297] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:25 2022] PUT /identity/v3/projects/7d5d6d15ce5d4c7692ba9dc3d4a262c5/users/eab68e3778bf4d1aa224cc9f46a36ebb/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:25.477859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.478415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.478923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.514894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] MFA Rules not processed for user `eab68e3778bf4d1aa224cc9f46a36ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:25.516768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:25.589594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1141/2298] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:25.601768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:25.603536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:25.628523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:25.666946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bf558-fa95-4b6b-891e-d8ed9606293c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:25.670674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819834b8-eb42-44be-b463-d4f43747ca7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:25.671371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819834b8-eb42-44be-b463-d4f43747ca7f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:25.671657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819834b8-eb42-44be-b463-d4f43747ca7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:25.671931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819834b8-eb42-44be-b463-d4f43747ca7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:25.682014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819834b8-eb42-44be-b463-d4f43747ca7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:25.683526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819834b8-eb42-44be-b463-d4f43747ca7f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:25.694176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1158/2299] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:38:26.155842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:26.230602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232ae4c-5c6b-4c97-9862-352c6fa01b76 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:26.241195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:26.242776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:26.243335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:26.243851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] PATH_INFO: `/v3/users/52cbd0ce79d04710873f8d39e8616b35` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:26.246111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52cbd0ce79d04710873f8d39e8616b35)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:26.249812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:26.303843 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:26.318999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52cbd0ce79d04710873f8d39e8616b35'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.328189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52cbd0ce79d04710873f8d39e8616b35'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.329497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52cbd0ce79d04710873f8d39e8616b35'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.337413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52cbd0ce79d04710873f8d39e8616b35'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.353145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52cbd0ce79d04710873f8d39e8616b35'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.359197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1142/2300] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:26 2022] DELETE /identity/v3/users/52cbd0ce79d04710873f8d39e8616b35 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:26.363917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:26.364430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:26.364535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:26.382493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:26.383778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:26.446941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1159/2301] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:26.458292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:26.460021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:26.526162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edc3e2-d38d-4d6c-8089-5674ba0def13 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:26.529866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:26.530517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:26.530799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:26.531091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:26.541084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:26.542534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:26.552473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1143/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:38:26.861702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:26.863807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:26.881342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b01398-85ac-4e9c-9366-cc7b6ee17a47 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:26.893385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:26.895166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:26.895530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:26.896132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] PATH_INFO: `/v3/projects/75436067372a4bf59034c620b1e4f4ba` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:26.897555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75436067372a4bf59034c620b1e4f4ba)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:26.900924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:26.963127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75436067372a4bf59034c620b1e4f4ba', 'request_id': 'req-df373855-b7da-4807-932d-218b94c512e6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.970402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75436067372a4bf59034c620b1e4f4ba', 'request_id': 'req-df373855-b7da-4807-932d-218b94c512e6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.979912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75436067372a4bf59034c620b1e4f4ba', 'request_id': 'req-df373855-b7da-4807-932d-218b94c512e6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.996927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] The token cache is being invalidate because project 75436067372a4bf59034c620b1e4f4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:26.997219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.997873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:26.999783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1160/2303] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:26 2022] DELETE /identity/v3/projects/75436067372a4bf59034c620b1e4f4ba => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:27.004640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:27.014783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011580e7-52a9-4661-b8af-27d7e3cf892b service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:27.075976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:27.076869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.077147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.077419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] PATH_INFO: `/v3/users/eab68e3778bf4d1aa224cc9f46a36ebb` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.078740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab68e3778bf4d1aa224cc9f46a36ebb)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:27.080726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:27.135045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab68e3778bf4d1aa224cc9f46a36ebb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.138712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:27.143906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab68e3778bf4d1aa224cc9f46a36ebb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.144813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab68e3778bf4d1aa224cc9f46a36ebb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.156498 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab68e3778bf4d1aa224cc9f46a36ebb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.170731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-06851fcf-ba4d-4d6d-82a0-9bcb5274843a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab68e3778bf4d1aa224cc9f46a36ebb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.178590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1144/2304] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:27 2022] DELETE /identity/v3/users/eab68e3778bf4d1aa224cc9f46a36ebb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:27.574843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:27.576798 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:27.595414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df373855-b7da-4807-932d-218b94c512e6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:27.652877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:27.677256 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:27.678257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.678632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.679024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] PATH_INFO: `/v3/projects/7d5d6d15ce5d4c7692ba9dc3d4a262c5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.681000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5d6d15ce5d4c7692ba9dc3d4a262c5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:27.683207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:27.736571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5d6d15ce5d4c7692ba9dc3d4a262c5', 'request_id': 'req-e737e353-f2db-4742-b0be-81c4cb4dc46e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.742946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5d6d15ce5d4c7692ba9dc3d4a262c5', 'request_id': 'req-e737e353-f2db-4742-b0be-81c4cb4dc46e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.748520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5d6d15ce5d4c7692ba9dc3d4a262c5', 'request_id': 'req-e737e353-f2db-4742-b0be-81c4cb4dc46e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.761796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] The token cache is being invalidate because project 7d5d6d15ce5d4c7692ba9dc3d4a262c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:27.762093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.762735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e737e353-f2db-4742-b0be-81c4cb4dc46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:27.764298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1161/2305] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:27 2022] DELETE /identity/v3/projects/7d5d6d15ce5d4c7692ba9dc3d4a262c5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:27.770611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.771178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.771178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.796889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:27.798155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:27.876619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1145/2306] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:27.887999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.888492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.888932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.915640 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:27.916761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:27.924694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1162/2307] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:27.929211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:27.939571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2e31b-05e5-4894-8458-7f0577320f08 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:27.944435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:27.945105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.945387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.945657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.946379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:27.946692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:27.948379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:27.963066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1146/2308] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:27.967828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:27.978075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a059-b345-412e-b4b2-4cd519226d6d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:27.982904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:27.983632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.983913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.984184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.985023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:27.987005 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.016862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1163/2309] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:27 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:28.018184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.028276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ae5d7-cde2-408f-9ea2-35208a08db6e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.089281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.089976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.090263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.090538 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.091336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.093420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.116934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1147/2310] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:28 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:28.121161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.131939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4c7ba-2be3-4a72-8e01-dcb268d06955 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.136758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.137469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.137757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.138288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.139032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.140693 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.152882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1164/2311] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:28.158040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.169186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93954b28-9e81-49fc-aa22-17b771778956 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.175177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc6ee2-456f-40ad-a8e8-1c2206f4aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.175858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc6ee2-456f-40ad-a8e8-1c2206f4aa87 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.176153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc6ee2-456f-40ad-a8e8-1c2206f4aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.176431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc6ee2-456f-40ad-a8e8-1c2206f4aa87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.177196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc6ee2-456f-40ad-a8e8-1c2206f4aa87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.178809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc6ee2-456f-40ad-a8e8-1c2206f4aa87 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.190288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1148/2312] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:28.195234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.206475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04953-69f2-4149-b111-6a429a823542 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.211308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.212263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.212503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.212780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] PATH_INFO: `/v3/projects/036f5c2fa2ba4404afe492e261a58671/users/a1c73ada95464bcba3d14c6678d84cf9/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.228246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036f5c2fa2ba4404afe492e261a58671, user_id=a1c73ada95464bcba3d14c6678d84cf9, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.231020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.256966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1165/2313] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:28 2022] PUT /identity/v3/projects/036f5c2fa2ba4404afe492e261a58671/users/a1c73ada95464bcba3d14c6678d84cf9/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:28.268587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.268905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.269186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.296163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] MFA Rules not processed for user `a1c73ada95464bcba3d14c6678d84cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:28.297447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:28.322906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:28.365456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1149/2314] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:28.370080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.436311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab179-08a0-43a1-bac7-7dd592c74b45 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.441123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.441789 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.442075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.442355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.443077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.445013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.465508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:28.474932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1166/2315] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:28 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:28.479572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.491534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ed5b7-3107-4b81-8e77-3af59b38872b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.547003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.547889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.548178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.548457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.549145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.551102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.571545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1150/2316] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:28 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:28.578177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.588832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ed1a7-de7c-4301-99fc-ec93c5552ff3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.594733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.595560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.596013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.596319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.597058 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.598688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.609844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1167/2317] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:28.614028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.624106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d70f7e-86df-41f9-b3ba-52cfed1b5a4d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.629037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.629915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.630195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.630470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] PATH_INFO: `/v3/projects/856275f2e3444b8488dc802f6c56fb09/users/cb27f3b1e5a741e889ee05ef7ea26a9a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.646026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856275f2e3444b8488dc802f6c56fb09, user_id=cb27f3b1e5a741e889ee05ef7ea26a9a, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.648705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.663292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1151/2318] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:28 2022] PUT /identity/v3/projects/856275f2e3444b8488dc802f6c56fb09/users/cb27f3b1e5a741e889ee05ef7ea26a9a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:28.667419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.678591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b817a1d-055d-4e5d-9971-ee44a9c92f63 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.743792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e3d19-5144-499d-8cef-0ef2759ddabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.744379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e3d19-5144-499d-8cef-0ef2759ddabe admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.744665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e3d19-5144-499d-8cef-0ef2759ddabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.744943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e3d19-5144-499d-8cef-0ef2759ddabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.745681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e3d19-5144-499d-8cef-0ef2759ddabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.747337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e3d19-5144-499d-8cef-0ef2759ddabe admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.758963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1168/2319] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:28.763292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.778397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487f47e-582c-4540-baee-9b4d0d6f4ac9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.783825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.784737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.785018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.785295 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] PATH_INFO: `/v3/projects/856275f2e3444b8488dc802f6c56fb09/users/cb27f3b1e5a741e889ee05ef7ea26a9a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.788255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856275f2e3444b8488dc802f6c56fb09, user_id=cb27f3b1e5a741e889ee05ef7ea26a9a, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.792677 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.803739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1152/2320] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:28 2022] PUT /identity/v3/projects/856275f2e3444b8488dc802f6c56fb09/users/cb27f3b1e5a741e889ee05ef7ea26a9a/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:28.814668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.814980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.815284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.841562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] MFA Rules not processed for user `cb27f3b1e5a741e889ee05ef7ea26a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:28.842760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:28.918341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1169/2321] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:28.930718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.932546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:28.998109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940645d-cfed-4ca0-a958-b5a00bab0380 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:29.002357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:29.003023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:29.003319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:29.003604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:29.012649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:29.014085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:29.024588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1153/2322] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:29.100056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:29.110818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bffe9f-bb7b-4ae2-9eb9-28545f276b56 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:29.114471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:29.115152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:29.115438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:29.115718 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:29.178041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:29.179683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:29.190126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1170/2323] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:30.054312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:30.055790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:30.078034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:30.138736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883d8db-d138-4939-9142-2c234aa5ef81 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:30.145694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:30.146414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:30.146701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:30.146980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:30.148875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:30.212964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:30.214561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd44655-7666-4bd1-8bf6-866cbfd58201 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:30.224702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1154/2324] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:38.520543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:38.529518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:38.584537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be62c-8af7-47d3-a548-6e765d8be794 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:38.589057 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:38.589930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:38.590209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:38.590479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] PATH_INFO: `/v3/users/a1c73ada95464bcba3d14c6678d84cf9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:38.591672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c73ada95464bcba3d14c6678d84cf9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:38.593614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:38.637774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c73ada95464bcba3d14c6678d84cf9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:38.646401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c73ada95464bcba3d14c6678d84cf9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:38.658307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c73ada95464bcba3d14c6678d84cf9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:38.658976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c73ada95464bcba3d14c6678d84cf9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:38.665903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c73ada95464bcba3d14c6678d84cf9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:38.672527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1171/2325] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:38 2022] DELETE /identity/v3/users/a1c73ada95464bcba3d14c6678d84cf9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:38.677321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:38.677919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:38.677919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:38.696144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:38.697221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:38.759703 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1155/2326] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:38.772746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:38.774279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:38.836126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566b6da-a5a3-4538-8d95-650f928660ea admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:38.839945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:38.840609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:38.840897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:38.841167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:38.851812 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:38.853271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:38.863978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1172/2327] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:39.226913 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.228386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:39.241406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765911aa-deea-46fb-ad05-52b55e52292a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.246837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.248756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.249048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.249326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] PATH_INFO: `/v3/projects/036f5c2fa2ba4404afe492e261a58671` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.251095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] RBAC: Authorizing `identity:delete_project(project_id=036f5c2fa2ba4404afe492e261a58671)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.253788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.312219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036f5c2fa2ba4404afe492e261a58671', 'request_id': 'req-e69986fb-dad3-4119-aa19-75a8647b377b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.315379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.322301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036f5c2fa2ba4404afe492e261a58671', 'request_id': 'req-e69986fb-dad3-4119-aa19-75a8647b377b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.330425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036f5c2fa2ba4404afe492e261a58671', 'request_id': 'req-e69986fb-dad3-4119-aa19-75a8647b377b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.340830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] The token cache is being invalidate because project 036f5c2fa2ba4404afe492e261a58671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:39.341158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.341981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.344222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1156/2328] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:39 2022] DELETE /identity/v3/projects/036f5c2fa2ba4404afe492e261a58671 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:39.348672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.358314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264558ee-8372-4cd2-abeb-ec283a6a766e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.384485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.417291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.418157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.418436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.418706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] PATH_INFO: `/v3/users/cb27f3b1e5a741e889ee05ef7ea26a9a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.419950 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb27f3b1e5a741e889ee05ef7ea26a9a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.421878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.477816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb27f3b1e5a741e889ee05ef7ea26a9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.487069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb27f3b1e5a741e889ee05ef7ea26a9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.499665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb27f3b1e5a741e889ee05ef7ea26a9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.500961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb27f3b1e5a741e889ee05ef7ea26a9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.508157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77cd4ccf-29aa-4791-b205-6c33cc422626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb27f3b1e5a741e889ee05ef7ea26a9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:39.514335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1173/2329] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:39 2022] DELETE /identity/v3/users/cb27f3b1e5a741e889ee05ef7ea26a9a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:40.014625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.017245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.031977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69986fb-dad3-4119-aa19-75a8647b377b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.111412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.112291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.112574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.112879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] PATH_INFO: `/v3/projects/856275f2e3444b8488dc802f6c56fb09` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.114120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=856275f2e3444b8488dc802f6c56fb09)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.116224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.168631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856275f2e3444b8488dc802f6c56fb09', 'request_id': 'req-af359366-6992-4476-93ba-b392a0ceca55'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:40.178888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856275f2e3444b8488dc802f6c56fb09', 'request_id': 'req-af359366-6992-4476-93ba-b392a0ceca55'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:40.186054 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856275f2e3444b8488dc802f6c56fb09', 'request_id': 'req-af359366-6992-4476-93ba-b392a0ceca55'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:40.197231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] The token cache is being invalidate because project 856275f2e3444b8488dc802f6c56fb09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:40.197525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:40.198201 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af359366-6992-4476-93ba-b392a0ceca55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:40.199848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1157/2330] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:40 2022] DELETE /identity/v3/projects/856275f2e3444b8488dc802f6c56fb09 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:40.204834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.205373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.205562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.225390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:40.226627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.310766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1174/2331] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:40.314561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.314906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.315153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.336083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:40.337316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.351263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1158/2332] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:40.353530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.364224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f209c-4fc8-4ed4-a1e4-b2e3c7dde27d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.368970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.369637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.369924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.370234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.370996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:40.371347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.373008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.386275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1175/2333] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:40.390570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.402810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bf7ed-de17-46d2-96be-74d642eb889d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.408189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.408814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.409090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.409608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.410067 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.412015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.447956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1159/2334] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:40 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:40.452016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.452658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.454198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.465896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbbdd-f38d-42b4-af0a-5b9122bb34b1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.518557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:40.524759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:40.583752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.584440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.584867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.585294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.585998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.588136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.589656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd703bd2-ba98-4d3a-ab8c-3ec48a09573b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.608609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.612238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.612565 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.613944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.616581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.629867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1176/2335] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:40 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:40.631436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.634512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.634692 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.636324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.649472 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1160/2336] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:40.650350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.657280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b0aa5-baf8-46b2-b479-cf765f6f38c8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.664952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b06648-5e93-4c82-ba48-bb0947e16a2e service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.669413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.670507 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.670627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.671172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.671684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.674341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.677978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.678391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.678790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.680530 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.682987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.705223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1161/2337] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:40.712281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.722940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98cff4-54f7-4b9b-8faa-9da1857f1b95 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.727979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.728666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.728949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.729225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.729964 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.732356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.744847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1162/2338] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:40.751126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.766957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b1141-dff9-49f4-864e-f18ab20bec77 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.771966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.773395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.773395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.773395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] PATH_INFO: `/v3/projects/2295c60dd5a749148022f4fe585217ca/users/3498b413bf4f4c8fb34b9eaa24e2f803/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.783234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.784826 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f301d9-c696-4014-a824-e06e24018ad4 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.790628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2295c60dd5a749148022f4fe585217ca, user_id=3498b413bf4f4c8fb34b9eaa24e2f803, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.793499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.796339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1177/2339] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:40.809339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1163/2340] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:40 2022] PUT /identity/v3/projects/2295c60dd5a749148022f4fe585217ca/users/3498b413bf4f4c8fb34b9eaa24e2f803/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:40.820432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.820741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.821021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.875025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] MFA Rules not processed for user `3498b413bf4f4c8fb34b9eaa24e2f803`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:40.877490 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:40.985519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1178/2341] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:40.986926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.103751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd0f19-d107-49e6-8b6c-032f221f6db7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.108926 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.110159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.110437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.110748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.111502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.113859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.147793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1164/2342] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:40 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:41.153259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.172509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687cc9c-7a3d-46ac-9234-a5bd3d173eda None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.257017 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:41.269819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.270650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.271009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.271389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.272216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.274392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.300537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1179/2343] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:41 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:41.301716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.312512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e2336-a349-4272-bb74-791519e0af91 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.319029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.319860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.320129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.320410 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.321252 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.323127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.335652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1165/2344] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:38:41.339391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.350653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e60142-fa69-4b01-a7cc-abe494e8492e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.356298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069f9dc-2261-4650-afbb-d1b42ca68e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.357513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069f9dc-2261-4650-afbb-d1b42ca68e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.357917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069f9dc-2261-4650-afbb-d1b42ca68e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.358292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069f9dc-2261-4650-afbb-d1b42ca68e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.359178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069f9dc-2261-4650-afbb-d1b42ca68e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.361567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069f9dc-2261-4650-afbb-d1b42ca68e03 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.382172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1180/2345] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:38:41.386428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.398792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1a92-9b47-4016-adcc-ad8fc0d4ef75 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.404474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.405393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.405669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.405942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] PATH_INFO: `/v3/projects/780707425e1745b692197a0bdcc39a5f/users/0320f2e5d9a549f18ccd1fc09b1e32ca/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.425343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780707425e1745b692197a0bdcc39a5f, user_id=0320f2e5d9a549f18ccd1fc09b1e32ca, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.428115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.445306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1166/2346] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:41 2022] PUT /identity/v3/projects/780707425e1745b692197a0bdcc39a5f/users/0320f2e5d9a549f18ccd1fc09b1e32ca/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:41.457612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796c116-bab4-4542-8860-7726f26704ae None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.457888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796c116-bab4-4542-8860-7726f26704ae None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.458168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796c116-bab4-4542-8860-7726f26704ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.494125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a796c116-bab4-4542-8860-7726f26704ae None None] MFA Rules not processed for user `0320f2e5d9a549f18ccd1fc09b1e32ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:41.495427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a796c116-bab4-4542-8860-7726f26704ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:41.571769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1181/2347] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:41.576833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.626930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:41.670089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb2ba-10b9-4024-a318-bd4ee2567193 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.676596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.677396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.677750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.678095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.679109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.681034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.723800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1167/2348] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:41 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:41.724985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796c116-bab4-4542-8860-7726f26704ae None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.738448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796c116-bab4-4542-8860-7726f26704ae None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.816897 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.817858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.818150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.818435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.819192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.821195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.845042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1182/2349] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:41 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:41.849089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.860455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c03f7-c08d-4274-a50c-73199569b5ac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.865306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.866011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.866294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.866767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.867550 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.869161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.881791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1168/2350] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:41.885831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.896686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0addee-3b72-40c9-ba36-06b20b0f99e8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:41.902187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:41.903525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:41.903659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:41.903822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] PATH_INFO: `/v3/projects/570a49c0e74e490c9afd3e53696f151e/users/cb035ae2329e4b8a9ad17eef29a7eb20/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:41.919448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=570a49c0e74e490c9afd3e53696f151e, user_id=cb035ae2329e4b8a9ad17eef29a7eb20, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:41.922667 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:41.940394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1183/2351] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:41 2022] PUT /identity/v3/projects/570a49c0e74e490c9afd3e53696f151e/users/cb035ae2329e4b8a9ad17eef29a7eb20/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:41.944769 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:41.954953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d98ca-e3f6-4538-966f-975bfdafdbef admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:42.018332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f426df-d789-4c9f-86b5-e397e6933594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:42.022844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f426df-d789-4c9f-86b5-e397e6933594 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:42.023288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f426df-d789-4c9f-86b5-e397e6933594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:42.023679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f426df-d789-4c9f-86b5-e397e6933594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:42.025220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f426df-d789-4c9f-86b5-e397e6933594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:42.027378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f426df-d789-4c9f-86b5-e397e6933594 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:42.043932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1169/2352] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:42.045542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:42.060336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c21abd-8f51-405a-a510-6a4b59538c0a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:42.067324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:42.068241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:42.068594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:42.068938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] PATH_INFO: `/v3/projects/570a49c0e74e490c9afd3e53696f151e/users/cb035ae2329e4b8a9ad17eef29a7eb20/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:42.072956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=570a49c0e74e490c9afd3e53696f151e, user_id=cb035ae2329e4b8a9ad17eef29a7eb20, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:42.074438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:42.089541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1184/2353] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:42 2022] PUT /identity/v3/projects/570a49c0e74e490c9afd3e53696f151e/users/cb035ae2329e4b8a9ad17eef29a7eb20/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:42.100023 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:42.100308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:42.100584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:42.142872 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] MFA Rules not processed for user `cb035ae2329e4b8a9ad17eef29a7eb20`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:42.144125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:42.221325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1170/2354] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:42.233624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:42.236905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:42.254363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:42.310510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367d962-3da7-41de-b2df-ae643ec2e82b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:42.315338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e8cc-82fe-439b-9ccc-603eddd67f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:42.316060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e8cc-82fe-439b-9ccc-603eddd67f09 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:42.316348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e8cc-82fe-439b-9ccc-603eddd67f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:42.316625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e8cc-82fe-439b-9ccc-603eddd67f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:42.331077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e8cc-82fe-439b-9ccc-603eddd67f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:42.332569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e8cc-82fe-439b-9ccc-603eddd67f09 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:42.353410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1185/2355] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:43.043439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:43.045274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:43.062099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4279f-33a1-4306-bb4e-3952cff0471a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:43.067161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:43.072059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.072580 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.073057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.109076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:43.149579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:43.151084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:43.162965 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1171/2356] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:43.702934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.702934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.704139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.733527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] MFA Rules not processed for user `c4284eb173fc49d1b059fdd2f3de2b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:43.734940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:43.794225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1186/2357] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:44.102761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.104559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:44.163362 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a9852-4afd-4fce-802f-b8ac14deb969 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.167184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.167839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.168150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.168429 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.170211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:44.180945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.182375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.191620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1172/2358] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:44.518893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.520491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:44.622250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09337c82-62f4-473f-bb41-5fabbc3a14cf None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.637415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.638108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.638573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.638854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.641763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:44.657774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.660053 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.673923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1187/2359] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:46.567424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:46.568985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:46.580636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ab8b-8c20-4a10-961c-1c9c0c2931f7 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:46.584701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:46.585356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:46.585647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:46.585922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:46.588076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:46.600154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:46.601664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:46.611683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1173/2360] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:47.473491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:47.483309 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197723b6-ed2c-4eb5-b8fd-ec240bb919bd service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:47.489448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f82c1-7977-4091-be0c-85fe5c1e6d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:47.490121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f82c1-7977-4091-be0c-85fe5c1e6d99 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:47.490402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f82c1-7977-4091-be0c-85fe5c1e6d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:47.490904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f82c1-7977-4091-be0c-85fe5c1e6d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:47.550843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:47.568152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f82c1-7977-4091-be0c-85fe5c1e6d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:47.569615 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f82c1-7977-4091-be0c-85fe5c1e6d99 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:47.579365 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1188/2361] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:49.806829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:49.881793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f917a21-1c11-4084-9039-e143d4dad5fb service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:49.886546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:49.887467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:49.887754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:49.888029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] PATH_INFO: `/v3/users/3498b413bf4f4c8fb34b9eaa24e2f803` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:49.889232 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3498b413bf4f4c8fb34b9eaa24e2f803)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:49.892469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:49.940761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3498b413bf4f4c8fb34b9eaa24e2f803'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:49.949111 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3498b413bf4f4c8fb34b9eaa24e2f803'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:49.949877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3498b413bf4f4c8fb34b9eaa24e2f803'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:49.952166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:49.959673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3498b413bf4f4c8fb34b9eaa24e2f803'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:49.972301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3498b413bf4f4c8fb34b9eaa24e2f803'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:49.978275 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1174/2362] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:49 2022] DELETE /identity/v3/users/3498b413bf4f4c8fb34b9eaa24e2f803 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:49.982725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:49.983221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:49.983331 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:50.003278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:50.004502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:50.070855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1189/2363] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:50.081170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:50.082621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:50.146493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092e614-e960-4789-8df1-6953f6733b74 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:50.150076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:50.150727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:50.151037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:50.151345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:50.161459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:50.162895 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:50.172676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1175/2364] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:50.503491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:50.505091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:50.528935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2a56a-6ca4-4481-a792-08be820e4686 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:50.536646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:50.537935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:50.538214 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:50.538657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] PATH_INFO: `/v3/projects/2295c60dd5a749148022f4fe585217ca` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:50.539928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2295c60dd5a749148022f4fe585217ca)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:50.541863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:50.609128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2295c60dd5a749148022f4fe585217ca', 'request_id': 'req-986b715c-9579-4fe0-b2e4-c799b747c864'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.619983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2295c60dd5a749148022f4fe585217ca', 'request_id': 'req-986b715c-9579-4fe0-b2e4-c799b747c864'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.628795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2295c60dd5a749148022f4fe585217ca', 'request_id': 'req-986b715c-9579-4fe0-b2e4-c799b747c864'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.648875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] The token cache is being invalidate because project 2295c60dd5a749148022f4fe585217ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:50.649218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.650037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-986b715c-9579-4fe0-b2e4-c799b747c864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.652184 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1190/2365] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:50 2022] DELETE /identity/v3/projects/2295c60dd5a749148022f4fe585217ca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:50.661382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:50.675594 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269b4d3-335f-4df7-b55d-52909d958b3c service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:50.770000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:50.771425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:50.771982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:50.772192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] PATH_INFO: `/v3/users/0320f2e5d9a549f18ccd1fc09b1e32ca` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:50.773676 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=0320f2e5d9a549f18ccd1fc09b1e32ca)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:50.776261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:50.835527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0320f2e5d9a549f18ccd1fc09b1e32ca'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.849229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0320f2e5d9a549f18ccd1fc09b1e32ca'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.850045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0320f2e5d9a549f18ccd1fc09b1e32ca'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.860142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0320f2e5d9a549f18ccd1fc09b1e32ca'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.878129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:50.879519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0320f2e5d9a549f18ccd1fc09b1e32ca'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:50.893356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1176/2366] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:50 2022] DELETE /identity/v3/users/0320f2e5d9a549f18ccd1fc09b1e32ca => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:51.158536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.159452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.160039 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.180929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:51.182173 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:51.246456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1191/2367] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:38:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:51.266917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:51.268708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:51.271350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:51.273263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:51.299594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:51.373289 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b81f-92e3-4dc1-8ac2-f540371a96ce admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:51.374210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629023e-01b6-4646-9d15-489fcebe547b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:51.379260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:51.379437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:51.380143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.380296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.380580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.380687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.380854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] PATH_INFO: `/v3/projects/780707425e1745b692197a0bdcc39a5f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.381126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.382033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780707425e1745b692197a0bdcc39a5f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:51.384000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:51.391108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:51.392819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:51.401331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1177/2368] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:38:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:38:51.444442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780707425e1745b692197a0bdcc39a5f', 'request_id': 'req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.452080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780707425e1745b692197a0bdcc39a5f', 'request_id': 'req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.458874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780707425e1745b692197a0bdcc39a5f', 'request_id': 'req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.473329 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] The token cache is being invalidate because project 780707425e1745b692197a0bdcc39a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:51.473619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.474259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.475943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1192/2369] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:51 2022] DELETE /identity/v3/projects/780707425e1745b692197a0bdcc39a5f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:51.480248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:51.491645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556c9f0-859b-46cd-8110-ca0b082eb932 service swift] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:51.598297 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:51.600791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.601284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.601773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] PATH_INFO: `/v3/users/cb035ae2329e4b8a9ad17eef29a7eb20` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.603940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb035ae2329e4b8a9ad17eef29a7eb20)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:51.607623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:51.715978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb035ae2329e4b8a9ad17eef29a7eb20'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.747448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb035ae2329e4b8a9ad17eef29a7eb20'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.748446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb035ae2329e4b8a9ad17eef29a7eb20'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.760359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb035ae2329e4b8a9ad17eef29a7eb20'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.777797 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb035ae2329e4b8a9ad17eef29a7eb20'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:51.785488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1178/2370] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:51 2022] DELETE /identity/v3/users/cb035ae2329e4b8a9ad17eef29a7eb20 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:52.510192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.512315 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:52.541406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.543606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:52.552108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.559642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e74b08-11ae-4c91-a4bf-19a5640c9cf6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.708994 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffac93-a28d-4e79-8f29-c193c87be302 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.710093 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.710975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.711285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.711584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] PATH_INFO: `/v3/projects/570a49c0e74e490c9afd3e53696f151e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.713502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=570a49c0e74e490c9afd3e53696f151e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.716727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.728674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.729401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.729832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.730128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.733467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:52.763393 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.766567 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772038f-8711-4de4-94fd-39449b3c098b service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.789012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1179/2371] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:38:52.839821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570a49c0e74e490c9afd3e53696f151e', 'request_id': 'req-6f82150d-6a6c-4a0b-a660-10ec45be2eed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:52.848278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.862665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570a49c0e74e490c9afd3e53696f151e', 'request_id': 'req-6f82150d-6a6c-4a0b-a660-10ec45be2eed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:52.882160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570a49c0e74e490c9afd3e53696f151e', 'request_id': 'req-6f82150d-6a6c-4a0b-a660-10ec45be2eed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:52.918147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] The token cache is being invalidate because project 570a49c0e74e490c9afd3e53696f151e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:52.918480 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:52.919845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6f82150d-6a6c-4a0b-a660-10ec45be2eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:52.925429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1193/2372] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:52 2022] DELETE /identity/v3/projects/570a49c0e74e490c9afd3e53696f151e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:52.931745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.932502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.932802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.966766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:52.968186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:53.121526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1180/2373] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:53.132401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.132727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.132978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.153616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:53.154837 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:53.163892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1194/2374] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:53.167837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.186894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c0606-152e-450e-b2e1-e14f0cad76f6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.193150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.193835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.194123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.194405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.195207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:53.195511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.198656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.213923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1181/2375] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:53.218230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.228869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863bd8-b50f-40fa-bbed-49d8afef1c84 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.235395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.236041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.236376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.237168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.237806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.240094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.280275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1195/2376] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:53 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:53.282307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.299760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed966-0b34-4ff8-804e-2a624adec968 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.382596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.383510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.383907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.384521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.385150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.387341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.417120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1182/2377] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:53 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:53.421143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.438591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d86a2f-0f35-421f-9209-9da85020d3e3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.444143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.445090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.445375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.445650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.446394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.448762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.464390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1196/2378] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:38:53.473504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.512884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72fd9d-e23d-4565-9d8f-0508096fe233 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.520633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd06eb-c662-4468-b29f-9500ccc4b618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.522391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd06eb-c662-4468-b29f-9500ccc4b618 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.522391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd06eb-c662-4468-b29f-9500ccc4b618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.522391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd06eb-c662-4468-b29f-9500ccc4b618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.523688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd06eb-c662-4468-b29f-9500ccc4b618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.526709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd06eb-c662-4468-b29f-9500ccc4b618 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.529820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:53.544363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1183/2379] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:53.548971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.567443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c554005-e83a-40ac-9b48-f2e644f0ce81 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.598819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.600106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.600586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.601106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] PATH_INFO: `/v3/projects/c61fb515f462494bbec8571798578715/users/6bb9497b3a4a452ab583661306d48aa7/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.651187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61fb515f462494bbec8571798578715, user_id=6bb9497b3a4a452ab583661306d48aa7, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.654851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.688565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1197/2380] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:53 2022] PUT /identity/v3/projects/c61fb515f462494bbec8571798578715/users/6bb9497b3a4a452ab583661306d48aa7/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:53.701792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.702125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.702394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.742578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] MFA Rules not processed for user `6bb9497b3a4a452ab583661306d48aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:53.744786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:53.821937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1184/2381] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:53.826239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.905146 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf8c4-49da-449b-925a-02af392ccf03 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.912088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.912783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.913082 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.913349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.914071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.916050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.943587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1198/2382] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:53 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:53.947613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.959826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3e143-4b91-4f13-89dc-fdb14f150f5b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:54.017167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:54.017974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.018328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.018607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.019363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:54.021265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:54.045935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1185/2383] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:53 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:54.050517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:54.061482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af9605-1da8-401d-9dd7-168652bbf0c6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:54.066665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:54.067379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.067658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.067927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.068653 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:54.070257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:54.082492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1199/2384] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:54.087967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:54.099675 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f356a5-e298-40a4-ac10-9acc69802adc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:54.105330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:54.106411 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.106518 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.106830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] PATH_INFO: `/v3/projects/4a9bdd4d929241d889588cdf5ee5ad96/users/1ec944a7b30a46b0b4aa2a57edbbde51/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.123671 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9bdd4d929241d889588cdf5ee5ad96, user_id=1ec944a7b30a46b0b4aa2a57edbbde51, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:54.126709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:54.143752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1186/2385] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:54 2022] PUT /identity/v3/projects/4a9bdd4d929241d889588cdf5ee5ad96/users/1ec944a7b30a46b0b4aa2a57edbbde51/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:54.147779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:54.158832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96b417-d52e-47ea-906f-31f9169eff56 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:54.206517 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:54.222688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53748cb1-eae0-4d52-81d4-e6f7596311a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:54.224113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53748cb1-eae0-4d52-81d4-e6f7596311a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.224113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53748cb1-eae0-4d52-81d4-e6f7596311a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.224314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53748cb1-eae0-4d52-81d4-e6f7596311a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.225023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53748cb1-eae0-4d52-81d4-e6f7596311a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:54.226654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53748cb1-eae0-4d52-81d4-e6f7596311a1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:54.237984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1200/2386] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:54.242308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:54.261748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44684c-9062-4b53-884c-58c0e0a26ba4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:54.274294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:54.275228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.275517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.275796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] PATH_INFO: `/v3/projects/4a9bdd4d929241d889588cdf5ee5ad96/users/1ec944a7b30a46b0b4aa2a57edbbde51/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.277949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9bdd4d929241d889588cdf5ee5ad96, user_id=1ec944a7b30a46b0b4aa2a57edbbde51, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:54.280617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:54.291348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1187/2387] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:54 2022] PUT /identity/v3/projects/4a9bdd4d929241d889588cdf5ee5ad96/users/1ec944a7b30a46b0b4aa2a57edbbde51/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:54.302012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.302300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.302581 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.335104 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] MFA Rules not processed for user `1ec944a7b30a46b0b4aa2a57edbbde51`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:54.336363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:54.420300 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1201/2388] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:54.431934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:54.434637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:54.502770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:54.525369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fb2f7-c330-4123-b7ce-af63896d7eae admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:54.529050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e625a-e4b1-4eaf-8cd9-495150e4b3dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:54.529700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e625a-e4b1-4eaf-8cd9-495150e4b3dc service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:54.529978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e625a-e4b1-4eaf-8cd9-495150e4b3dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:54.530259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e625a-e4b1-4eaf-8cd9-495150e4b3dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:54.543514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e625a-e4b1-4eaf-8cd9-495150e4b3dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:54.545417 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e625a-e4b1-4eaf-8cd9-495150e4b3dc service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:54.560037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1188/2389] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:55.384881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:55.528854 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c60f5-0a9d-44bb-a865-0f857d0d5916 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:55.534876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:55.539220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:55.539220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:55.539220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] PATH_INFO: `/v3/users/6bb9497b3a4a452ab583661306d48aa7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:55.540200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb9497b3a4a452ab583661306d48aa7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:55.542310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:55.619337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb9497b3a4a452ab583661306d48aa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:55.627335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb9497b3a4a452ab583661306d48aa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:55.641844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb9497b3a4a452ab583661306d48aa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:55.642998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb9497b3a4a452ab583661306d48aa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:55.647461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:55.662817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb9497b3a4a452ab583661306d48aa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:55.677477 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1202/2390] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:55 2022] DELETE /identity/v3/users/6bb9497b3a4a452ab583661306d48aa7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:55.682346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:55.682854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:55.682951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:55.703326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:55.704647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:55.788901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1189/2391] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:55.802478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:55.804059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:55.893125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b489b-2a83-466e-93c3-9b19b71a651d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:55.900268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:55.901220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:55.901486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:55.901763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:55.913538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:55.915022 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:55.929792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1203/2392] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:56.359250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:56.361636 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:56.375153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcfc70-ca0e-48a1-a005-19a4659fa72f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:56.388450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:56.389350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:56.389633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:56.389914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] PATH_INFO: `/v3/projects/c61fb515f462494bbec8571798578715` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:56.396348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61fb515f462494bbec8571798578715)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:56.396348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:56.488858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61fb515f462494bbec8571798578715', 'request_id': 'req-503aac36-7030-44d7-9299-40017f0c0e98'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.500563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61fb515f462494bbec8571798578715', 'request_id': 'req-503aac36-7030-44d7-9299-40017f0c0e98'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.508914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61fb515f462494bbec8571798578715', 'request_id': 'req-503aac36-7030-44d7-9299-40017f0c0e98'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.523088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] The token cache is being invalidate because project c61fb515f462494bbec8571798578715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:56.523432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.524103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.526068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1190/2393] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:56 2022] DELETE /identity/v3/projects/c61fb515f462494bbec8571798578715 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:56.531235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:56.545974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6759a03-8ab3-4943-ac2c-3c3b49c9cfbf service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:56.663180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:56.665019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:56.665019 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:56.665193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] PATH_INFO: `/v3/users/1ec944a7b30a46b0b4aa2a57edbbde51` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:56.666739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec944a7b30a46b0b4aa2a57edbbde51)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:56.669074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:56.755582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec944a7b30a46b0b4aa2a57edbbde51'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.764930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec944a7b30a46b0b4aa2a57edbbde51'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.781864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec944a7b30a46b0b4aa2a57edbbde51'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.782838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec944a7b30a46b0b4aa2a57edbbde51'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.791484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8c8d6a98-5e21-418b-b862-5140973e52cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec944a7b30a46b0b4aa2a57edbbde51'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:56.802846 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1204/2394] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:56 2022] DELETE /identity/v3/users/1ec944a7b30a46b0b4aa2a57edbbde51 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:57.188608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:57.191224 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:57.241419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aac36-7030-44d7-9299-40017f0c0e98 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:57.329444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:57.331037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.331604 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.332171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] PATH_INFO: `/v3/projects/4a9bdd4d929241d889588cdf5ee5ad96` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.333709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9bdd4d929241d889588cdf5ee5ad96)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:57.336113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:57.343582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:57.428069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9bdd4d929241d889588cdf5ee5ad96', 'request_id': 'req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:57.441141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9bdd4d929241d889588cdf5ee5ad96', 'request_id': 'req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:57.449131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9bdd4d929241d889588cdf5ee5ad96', 'request_id': 'req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:57.474007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] The token cache is being invalidate because project 4a9bdd4d929241d889588cdf5ee5ad96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:57.474336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:57.474958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c7cbd7c5-bdeb-446f-95dd-716120fda4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:57.477367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1191/2395] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:57 2022] DELETE /identity/v3/projects/4a9bdd4d929241d889588cdf5ee5ad96 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:57.481930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.482321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.482724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.519169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:57.520555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:57.528160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:57.629981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1205/2396] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:57.637503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.637800 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.638084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.664601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:57.665866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:57.674796 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1192/2397] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:57.680046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:57.691537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b23d61-054e-4a5f-aabc-f5ad4b0dd8eb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:57.704263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:57.705179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.705461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.705731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.706474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:57.706779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:57.708523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:57.732002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:57.735187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1206/2398] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:57.744430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38d13a-19d2-41c4-a740-31f6bb6f156a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:57.749881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:57.750562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.750853 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.751156 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.752341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:57.753786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:57.791369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1193/2399] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:57 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:38:57.795744 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:57.812364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b11b-339b-4bb0-b4fb-db2f834cc833 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:57.894001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:57.894791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.895190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.895513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.896394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:57.898723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:57.929869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1207/2400] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:57 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:57.932052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:57.959540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ba54a-8c9b-44b2-9f2d-94984ad133ff admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:57.970177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:57.970870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:57.971181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:57.971460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:57.972200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:57.981324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:58.016889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1194/2401] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:58.025204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:58.041686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641941a-ae90-410b-b56f-1d1dbb6449fc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:58.048810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56263c36-87fa-4d90-b748-c72b04378097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:58.049499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56263c36-87fa-4d90-b748-c72b04378097 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.050832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56263c36-87fa-4d90-b748-c72b04378097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.051884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56263c36-87fa-4d90-b748-c72b04378097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.052016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56263c36-87fa-4d90-b748-c72b04378097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:58.054140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56263c36-87fa-4d90-b748-c72b04378097 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:58.069826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:58.070270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1208/2402] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:38:58.084939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34719fa3-ab70-4a56-b34f-a607ef59f4c0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:58.089951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:58.091122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.091496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.091849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] PATH_INFO: `/v3/projects/f9a8487cc51a4fd3b2f6c57593802174/users/e0159ebb327e48568dd4e7989c501c70/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.113126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a8487cc51a4fd3b2f6c57593802174, user_id=e0159ebb327e48568dd4e7989c501c70, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:58.119266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:58.143711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1195/2403] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:58 2022] PUT /identity/v3/projects/f9a8487cc51a4fd3b2f6c57593802174/users/e0159ebb327e48568dd4e7989c501c70/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:58.206962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.207326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.207573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.240864 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] MFA Rules not processed for user `e0159ebb327e48568dd4e7989c501c70`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:58.242314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:58.381928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1209/2404] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:58.395365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:58.397330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:58.483331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1dfae-00a4-4352-889b-c14964a8ac6c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:58.493175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d740-6dba-42cf-b643-ad671e6c28a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:58.493905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d740-6dba-42cf-b643-ad671e6c28a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.494191 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d740-6dba-42cf-b643-ad671e6c28a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.494465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d740-6dba-42cf-b643-ad671e6c28a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.505984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d740-6dba-42cf-b643-ad671e6c28a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:58.507581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d740-6dba-42cf-b643-ad671e6c28a8 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:58.521228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1196/2405] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:58.548127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:58.557834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:58.653556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c048c2-d433-47e9-b71e-c8b9b49f50e5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:58.661997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:58.662924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.663241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.663524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] PATH_INFO: `/v3/users/e0159ebb327e48568dd4e7989c501c70` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.667163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0159ebb327e48568dd4e7989c501c70)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:58.669498 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:58.791403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0159ebb327e48568dd4e7989c501c70'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:58.838658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0159ebb327e48568dd4e7989c501c70'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:58.853347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0159ebb327e48568dd4e7989c501c70'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:58.854227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0159ebb327e48568dd4e7989c501c70'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:58.862809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0159ebb327e48568dd4e7989c501c70'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:58.870196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1210/2406] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:58 2022] DELETE /identity/v3/users/e0159ebb327e48568dd4e7989c501c70 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:58.874865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.875969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.875969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.904200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:58.906087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:58.943421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:58.993826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1197/2407] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:59.001160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:59.003006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:59.133637 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50451906-b7b5-4ba8-91ba-d2c2adbb7269 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:59.147362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff319d-f2de-4f73-bf37-2d6b6f3e4d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:59.148570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff319d-f2de-4f73-bf37-2d6b6f3e4d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:59.149233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff319d-f2de-4f73-bf37-2d6b6f3e4d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:59.149526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff319d-f2de-4f73-bf37-2d6b6f3e4d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:59.170691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff319d-f2de-4f73-bf37-2d6b6f3e4d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:59.173279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff319d-f2de-4f73-bf37-2d6b6f3e4d61 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:59.186518 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1211/2408] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:59.542584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:59.544228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:59.575237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a51ab-5071-45e6-ac8e-b35e885e41cf None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:59.590027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:59.592200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:59.592743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:59.593776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] PATH_INFO: `/v3/projects/f9a8487cc51a4fd3b2f6c57593802174` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:59.597501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a8487cc51a4fd3b2f6c57593802174)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:59.605415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:59.710046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a8487cc51a4fd3b2f6c57593802174', 'request_id': 'req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:59.723407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a8487cc51a4fd3b2f6c57593802174', 'request_id': 'req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:59.736914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a8487cc51a4fd3b2f6c57593802174', 'request_id': 'req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:59.758915 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] The token cache is being invalidate because project f9a8487cc51a4fd3b2f6c57593802174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:59.759761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:59.760919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a0b3e34-4667-44b4-a9f5-cec7f296a97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:59.769040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1198/2409] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:59 2022] DELETE /identity/v3/projects/f9a8487cc51a4fd3b2f6c57593802174 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:59.771282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:59.771282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:59.771890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:59.804600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:59.807457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:59.837730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:59.885988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1212/2410] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:59.895037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:59.895782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:59.896272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:59.936032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:59.937725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:59.954085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1199/2411] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:59.957696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:59.976765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae59d74-792c-47f7-b1fb-c9cd023e9543 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.005024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.005727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.006013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.006293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.007087 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:00.007557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.009374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.064303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1213/2412] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:00.065379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.076848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985a2cf-48d6-48b2-ab9b-b3fafb4cb93b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.085610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.085814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.086131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.086478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.087725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.089668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.129839 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1200/2413] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:00 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:00.133860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.151772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089def1-da5b-45f2-a0cb-a5ce92b7779c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.254655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.256216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.256606 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.257002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.258847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.261081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.307210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1214/2414] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:00 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:00.311415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.331424 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ee73-449f-412e-8927-49f328a85bfd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.337072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.337761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.338114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.338501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.339375 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.341881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.377438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1201/2415] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:00.381928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.401905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b3c8-3f95-466a-9ba3-d6b5773a768c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.412220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89a394-f9a7-4cc3-879c-bef72a1d793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.413879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89a394-f9a7-4cc3-879c-bef72a1d793f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.415134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89a394-f9a7-4cc3-879c-bef72a1d793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.416346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89a394-f9a7-4cc3-879c-bef72a1d793f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.417363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89a394-f9a7-4cc3-879c-bef72a1d793f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.419452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89a394-f9a7-4cc3-879c-bef72a1d793f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.431994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1215/2416] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:00.436535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.453744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a56d91-a9f4-4a3c-b010-269791d33300 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.463154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.464906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.465278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.466277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] PATH_INFO: `/v3/projects/9fae342fccbb4bc7b869bc76e36d8354/users/2d05d9e31fd6402d9db5b4168ce2d4e1/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.487726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fae342fccbb4bc7b869bc76e36d8354, user_id=2d05d9e31fd6402d9db5b4168ce2d4e1, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.490868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.508460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1202/2417] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:00 2022] PUT /identity/v3/projects/9fae342fccbb4bc7b869bc76e36d8354/users/2d05d9e31fd6402d9db5b4168ce2d4e1/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:00.523877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.524848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.525598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.596250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] MFA Rules not processed for user `2d05d9e31fd6402d9db5b4168ce2d4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:00.597790 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:00.718656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1216/2418] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:00.743188 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.745834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:00.867581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7aad-592d-4710-babd-86eb77efb5c1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.875995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afe39c-d9d7-4f47-b54b-8c98879f09a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.877167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afe39c-d9d7-4f47-b54b-8c98879f09a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.877776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afe39c-d9d7-4f47-b54b-8c98879f09a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.878233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afe39c-d9d7-4f47-b54b-8c98879f09a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.893921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:00.898105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afe39c-d9d7-4f47-b54b-8c98879f09a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.903817 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afe39c-d9d7-4f47-b54b-8c98879f09a9 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.926058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1203/2419] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:08.714489 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:08.742574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:08.781035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4df9a1-e1aa-40c5-80d2-25ed70306794 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:08.785608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:08.786479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:08.786773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:08.787050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] PATH_INFO: `/v3/users/2d05d9e31fd6402d9db5b4168ce2d4e1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:08.788265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d05d9e31fd6402d9db5b4168ce2d4e1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:08.790867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:08.838835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d05d9e31fd6402d9db5b4168ce2d4e1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:08.847138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d05d9e31fd6402d9db5b4168ce2d4e1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:08.859523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d05d9e31fd6402d9db5b4168ce2d4e1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:08.860318 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d05d9e31fd6402d9db5b4168ce2d4e1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:08.867894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d05d9e31fd6402d9db5b4168ce2d4e1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:08.873946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1217/2420] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:08 2022] DELETE /identity/v3/users/2d05d9e31fd6402d9db5b4168ce2d4e1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:08.878460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:08.878987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:08.879124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:08.896592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:08.897805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:08.957810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1204/2421] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:08.968226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:08.969504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:09.025969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b992c88-2db3-4fba-ab01-a6b741f2c228 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.029822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1648d-b6dd-4718-a4bd-352084136779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.031414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1648d-b6dd-4718-a4bd-352084136779 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.031723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1648d-b6dd-4718-a4bd-352084136779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.031997 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1648d-b6dd-4718-a4bd-352084136779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.041914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1648d-b6dd-4718-a4bd-352084136779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.043389 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1648d-b6dd-4718-a4bd-352084136779 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.054576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1218/2422] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:39:09.352216 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.353852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:09.368998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323a892-e058-425d-82e0-1e6b38ce9954 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.374609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.375534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.375956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.376087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] PATH_INFO: `/v3/projects/9fae342fccbb4bc7b869bc76e36d8354` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.377319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fae342fccbb4bc7b869bc76e36d8354)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.379359 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.426332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fae342fccbb4bc7b869bc76e36d8354', 'request_id': 'req-250d21d8-c144-431c-a5bc-f2e1edee6bf1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:09.434717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fae342fccbb4bc7b869bc76e36d8354', 'request_id': 'req-250d21d8-c144-431c-a5bc-f2e1edee6bf1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:09.442031 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fae342fccbb4bc7b869bc76e36d8354', 'request_id': 'req-250d21d8-c144-431c-a5bc-f2e1edee6bf1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:09.453770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] The token cache is being invalidate because project 9fae342fccbb4bc7b869bc76e36d8354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:09.454071 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:09.454743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-250d21d8-c144-431c-a5bc-f2e1edee6bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:09.456382 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1205/2423] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:09 2022] DELETE /identity/v3/projects/9fae342fccbb4bc7b869bc76e36d8354 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:09.461294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.461591 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.461876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.479954 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:09.481076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:09.524700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:09.542766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1219/2424] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:09.550921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.551248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.551533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.570741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:09.572123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:09.581380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1206/2425] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:09.585772 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.596343 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420ccec-99e3-4317-bf7d-adc271c990f1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.611535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.612215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.612577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.612917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.613729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:09.614226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.615943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.629773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1220/2426] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:09.634279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.645428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae81e9-2226-4937-9f9a-ad0847f13b3b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.650891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.651614 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.651901 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.652185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.652890 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.655323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.692560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1207/2427] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:09 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:09.693719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.704157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143e5a-80bc-440d-9686-db2cc18e905a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.776111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.776830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.777097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.777372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.778371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.780963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.818369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1221/2428] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:09 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:39:09.823211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.838061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79665f0-00d9-4490-a0d2-c04e388d196f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.843730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.844571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.844933 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.845350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.846162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.848000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.860491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1208/2429] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:09.864526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.874689 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba234ff-0501-4a2b-b392-5a53a3a4612b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.879457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deb624-aa61-48a1-9d36-eb7e9028ef85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.880120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deb624-aa61-48a1-9d36-eb7e9028ef85 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.880407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deb624-aa61-48a1-9d36-eb7e9028ef85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.880685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deb624-aa61-48a1-9d36-eb7e9028ef85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.881405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deb624-aa61-48a1-9d36-eb7e9028ef85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.883016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deb624-aa61-48a1-9d36-eb7e9028ef85 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.893962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1222/2430] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:39:09.898229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:09.909496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadd966-3986-48cd-92fe-fb892ffd9005 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:09.914149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:09.915028 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.915341 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.915620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] PATH_INFO: `/v3/projects/c0b1f89c9d104bcd8acbb075270cfae3/users/69059990cf474e46825c534220b3c3ae/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.929888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b1f89c9d104bcd8acbb075270cfae3, user_id=69059990cf474e46825c534220b3c3ae, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:09.932581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:09.945896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1209/2431] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:09 2022] PUT /identity/v3/projects/c0b1f89c9d104bcd8acbb075270cfae3/users/69059990cf474e46825c534220b3c3ae/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:09.957454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:09.957769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:09.958046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:09.984982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] MFA Rules not processed for user `69059990cf474e46825c534220b3c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:09.986191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:10.055696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1223/2432] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:10.060170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.078946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:10.153018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c15a2-a695-479d-b88a-c9df44e4fead admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.159187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.160199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.160199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.160446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.161145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.163149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.193060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1210/2433] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:10 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:10.197311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.209842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b26593-07f1-41bc-96e9-d6848eb236a0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.257674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:10.276953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.277666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.277955 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.278235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.278945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.281353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.310696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1224/2434] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:10 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:39:10.313387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.331024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142de62-1366-45d8-a9ec-1ad4bd86b7ce admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.350337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.351173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.351606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.351902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.352647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.354303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.370015 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1211/2435] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:10.375342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.386455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03f19c-8dad-41ac-ac69-46c773ba92f9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.393425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740599d-475d-49d0-876e-12dc50ebd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.394259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740599d-475d-49d0-876e-12dc50ebd48d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.394651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740599d-475d-49d0-876e-12dc50ebd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.395032 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740599d-475d-49d0-876e-12dc50ebd48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.396107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740599d-475d-49d0-876e-12dc50ebd48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.397874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740599d-475d-49d0-876e-12dc50ebd48d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.410412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1225/2436] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:10.414865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.427425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e58e96-5843-49ae-ab0b-ea14227345d5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.435277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.437247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.437705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.438227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] PATH_INFO: `/v3/projects/502fc11047564050865f3fededb4b60d/users/aa10eed78af84c79b3e9ab12e77046ba/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.462762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=502fc11047564050865f3fededb4b60d, user_id=aa10eed78af84c79b3e9ab12e77046ba, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.465645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.480010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1212/2437] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:10 2022] PUT /identity/v3/projects/502fc11047564050865f3fededb4b60d/users/aa10eed78af84c79b3e9ab12e77046ba/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:10.491966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.492264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.492550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.525381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] MFA Rules not processed for user `aa10eed78af84c79b3e9ab12e77046ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:10.526625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:10.596209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1226/2438] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:10.600768 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.674042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed36528-8079-41ba-a2fb-2e747b481ead admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.682324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.683514 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.683919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.684425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.685833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.693082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.720619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1213/2439] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:10 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:10.724496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.738634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fae98-78bc-464e-8844-b9b0bb4089d5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.798387 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.799164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.799388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.799657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.800367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.802373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.826600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1227/2440] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:10 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:10.832608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.845226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cce8e-e853-4799-8e75-7a7f5365f7be admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.851364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.852075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.852364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.852646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.853391 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.855016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.867936 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1214/2441] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:10.871994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.884787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fbd60-d32e-4cd2-84ca-b08a09ff0a3c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:10.901388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:10.902292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:10.902575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:10.902850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] PATH_INFO: `/v3/projects/7363d5b6ec744070952ae8f25770301c/users/88c99584e1c5413fab619af8a2a901bc/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:10.922199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7363d5b6ec744070952ae8f25770301c, user_id=88c99584e1c5413fab619af8a2a901bc, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:10.924930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:10.946399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1228/2442] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:10 2022] PUT /identity/v3/projects/7363d5b6ec744070952ae8f25770301c/users/88c99584e1c5413fab619af8a2a901bc/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:10.947399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:10.958189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f3f86-5d73-4ded-ab72-ec772c24af00 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:11.012517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:11.045859 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd4c75-89bf-4393-a90a-3719cb4b24f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:11.046704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd4c75-89bf-4393-a90a-3719cb4b24f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:11.046998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd4c75-89bf-4393-a90a-3719cb4b24f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:11.047307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd4c75-89bf-4393-a90a-3719cb4b24f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:11.048141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd4c75-89bf-4393-a90a-3719cb4b24f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:11.049780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd4c75-89bf-4393-a90a-3719cb4b24f2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:11.072663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:11.080290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1215/2443] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:39:11.085999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53585593-b120-4c8c-8635-a2d1ca9689dd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:11.105149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:11.106079 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:11.106358 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:11.106633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] PATH_INFO: `/v3/projects/7363d5b6ec744070952ae8f25770301c/users/88c99584e1c5413fab619af8a2a901bc/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:11.109934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7363d5b6ec744070952ae8f25770301c, user_id=88c99584e1c5413fab619af8a2a901bc, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:11.123403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:11.159363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1229/2444] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:11 2022] PUT /identity/v3/projects/7363d5b6ec744070952ae8f25770301c/users/88c99584e1c5413fab619af8a2a901bc/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:11.170162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:11.170845 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:11.171316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:11.212093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] MFA Rules not processed for user `88c99584e1c5413fab619af8a2a901bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:11.213868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:11.321633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1216/2445] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:11.329738 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:11.332138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:11.339218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:11.404496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea486fb-610c-4212-8d76-4580e355677c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:11.409308 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:11.410074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:11.410427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:11.410763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:11.501641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:11.503362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:11.520566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1230/2446] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:39:13.212946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:13.214243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:13.226404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9556-e3de-437a-8959-e9f456eca894 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:13.230578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c8b1b-ef6d-4d71-a659-d8e23a9046a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:13.231332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c8b1b-ef6d-4d71-a659-d8e23a9046a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:13.231621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c8b1b-ef6d-4d71-a659-d8e23a9046a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:13.231902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c8b1b-ef6d-4d71-a659-d8e23a9046a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:13.304524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c8b1b-ef6d-4d71-a659-d8e23a9046a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:13.306130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c8b1b-ef6d-4d71-a659-d8e23a9046a7 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:13.316898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1217/2447] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:13.975867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:13.986666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe402b82-b3b2-4beb-ac5e-f2d970f2cc42 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:13.992771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:13.993858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:13.993858 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:13.994366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:14.005055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:14.006752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:14.018912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1231/2448] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:14.488683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:14.489194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:14.489312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:14.507733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:14.508949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:14.567522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1218/2449] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:39:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:39:14.579961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:14.581406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:14.651018 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd90bd-2d7a-4f5a-8835-74eb3f9358ea service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:14.657461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:14.658158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:14.658400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:14.658673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:14.667462 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:14.669070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:14.679441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1232/2450] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:39:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:39:16.547577 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:16.549112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:16.555858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:16.648086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534b377-bf18-4c80-8d41-43e602ace0e1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:16.652273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:16.653229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:16.653740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:16.654260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:16.657447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:16.753187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:16.754956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d77ef-d7ed-4603-b862-56798df05df0 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:16.767717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1219/2451] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:18.139383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:18.162940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5930aca-9a61-453f-978c-f1d87633db58 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:18.172260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:18.174661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:18.179592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:18.180031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] PATH_INFO: `/v3/users/69059990cf474e46825c534220b3c3ae` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:18.182852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69059990cf474e46825c534220b3c3ae)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:18.184334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:18.236108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:18.273877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69059990cf474e46825c534220b3c3ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:18.304914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69059990cf474e46825c534220b3c3ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:18.323625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69059990cf474e46825c534220b3c3ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:18.324526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69059990cf474e46825c534220b3c3ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:18.334671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69059990cf474e46825c534220b3c3ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:18.345669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1233/2452] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:18 2022] DELETE /identity/v3/users/69059990cf474e46825c534220b3c3ae => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:18.350743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:18.351610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:18.352119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:18.393042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:18.394575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:18.468218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1220/2453] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:18.479121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:18.480907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:18.608101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837c3ce-ddb5-4e3f-b84a-8c33193b0be7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:18.613775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:18.614514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:18.614863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:18.615341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:18.628630 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:18.630144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:18.642281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1234/2454] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:19.068087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:19.069763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:19.112649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99230-7eb3-42db-93b6-117def91b2f1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:19.125131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:19.145545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:19.145881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:19.146115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] PATH_INFO: `/v3/projects/c0b1f89c9d104bcd8acbb075270cfae3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:19.147884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b1f89c9d104bcd8acbb075270cfae3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:19.149978 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:19.243792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b1f89c9d104bcd8acbb075270cfae3', 'request_id': 'req-83bbe721-2d55-48f0-99c5-a0553c42e713'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.246632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:19.255887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b1f89c9d104bcd8acbb075270cfae3', 'request_id': 'req-83bbe721-2d55-48f0-99c5-a0553c42e713'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.271746 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b1f89c9d104bcd8acbb075270cfae3', 'request_id': 'req-83bbe721-2d55-48f0-99c5-a0553c42e713'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.288908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] The token cache is being invalidate because project c0b1f89c9d104bcd8acbb075270cfae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:19.289287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.291336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.293278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1221/2455] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:19 2022] DELETE /identity/v3/projects/c0b1f89c9d104bcd8acbb075270cfae3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:19.297708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:19.322736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fd29d-0fbd-4684-a0aa-406685dd678c service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:19.449398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:19.450294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:19.451189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:19.452014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] PATH_INFO: `/v3/users/aa10eed78af84c79b3e9ab12e77046ba` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:19.453631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa10eed78af84c79b3e9ab12e77046ba)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:19.455940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:19.575809 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa10eed78af84c79b3e9ab12e77046ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.583422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:19.604392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa10eed78af84c79b3e9ab12e77046ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.674608 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa10eed78af84c79b3e9ab12e77046ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.676233 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa10eed78af84c79b3e9ab12e77046ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.684771 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa10eed78af84c79b3e9ab12e77046ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:19.695919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1235/2456] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:19 2022] DELETE /identity/v3/users/aa10eed78af84c79b3e9ab12e77046ba => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:20.089493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:20.092611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:20.116167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbe721-2d55-48f0-99c5-a0553c42e713 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:20.294354 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:20.296396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:20.296914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:20.297405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] PATH_INFO: `/v3/projects/502fc11047564050865f3fededb4b60d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:20.298951 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=502fc11047564050865f3fededb4b60d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:20.301197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:20.406355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502fc11047564050865f3fededb4b60d', 'request_id': 'req-d455012b-163e-46e4-a8d2-e97d50cfea5f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.418993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502fc11047564050865f3fededb4b60d', 'request_id': 'req-d455012b-163e-46e4-a8d2-e97d50cfea5f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.433027 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502fc11047564050865f3fededb4b60d', 'request_id': 'req-d455012b-163e-46e4-a8d2-e97d50cfea5f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.451303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] The token cache is being invalidate because project 502fc11047564050865f3fededb4b60d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:20.452076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.453099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.475371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1222/2457] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:20 2022] DELETE /identity/v3/projects/502fc11047564050865f3fededb4b60d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:20.478961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:20.480531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:20.497179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835ad0b-bf10-4ed8-8c09-3c921f9b4924 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:20.594773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:20.595802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:20.596161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:20.596592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] PATH_INFO: `/v3/users/88c99584e1c5413fab619af8a2a901bc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:20.597939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c99584e1c5413fab619af8a2a901bc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:20.600154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:20.690709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c99584e1c5413fab619af8a2a901bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.725221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c99584e1c5413fab619af8a2a901bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.752348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c99584e1c5413fab619af8a2a901bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.753366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c99584e1c5413fab619af8a2a901bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.764845 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85322374-3554-443b-964d-be50b70c5c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c99584e1c5413fab619af8a2a901bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:20.774179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1236/2458] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:20 2022] DELETE /identity/v3/users/88c99584e1c5413fab619af8a2a901bc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:21.270722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:21.270722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:21.295602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455012b-163e-46e4-a8d2-e97d50cfea5f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:21.458340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:21.459366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:21.459725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:21.460074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] PATH_INFO: `/v3/projects/7363d5b6ec744070952ae8f25770301c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:21.461456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7363d5b6ec744070952ae8f25770301c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:21.463575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:21.489833 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:21.584544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7363d5b6ec744070952ae8f25770301c', 'request_id': 'req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:21.600149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7363d5b6ec744070952ae8f25770301c', 'request_id': 'req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:21.611575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7363d5b6ec744070952ae8f25770301c', 'request_id': 'req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:21.640710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] The token cache is being invalidate because project 7363d5b6ec744070952ae8f25770301c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:21.641840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:21.643449 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b70d0f1e-c07c-43c4-8c4e-bb90eb0e2386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:21.648562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1223/2459] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:21 2022] DELETE /identity/v3/projects/7363d5b6ec744070952ae8f25770301c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:21.653532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:21.654139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:21.654348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:21.696195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:21.700563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:21.810840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:21.850095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1237/2460] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:21.859400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:21.859787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:21.860143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:21.898559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:21.899947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:21.930908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1224/2461] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:21.937796 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:21.973765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77256dd4-2ce1-4107-8fdb-780604834e34 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:21.996152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:21.997473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:21.999621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.006294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.007657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:22.008338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.010178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.039695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1238/2462] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:22.044190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.073852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.076208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:22.076476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b6176-876d-44e5-ab06-fa3f2e4ab3fa None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.096163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.097840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.098969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.101032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.102587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.105556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.163343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1225/2463] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:22 2022] POST /identity/v3/projects => generated 399 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:22.171893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.282180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bd146-9906-4781-9542-1f618cb56e8f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.341247 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a9bb8-9cf9-48cb-a361-b6eff708d1a7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.352561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.353340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.353717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.353919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.354723 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.357087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.392935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1226/2464] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:22 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:22.392935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.399542 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:22.406762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.407900 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.408416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.408903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.412341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:22.417384 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e03df6-f823-4f4b-a9d0-4a253d4434d6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.424607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.425512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.425793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.426069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] PATH_INFO: `/v3/users/1ee22783b3234c4f905fedce7d59f01d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.428055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee22783b3234c4f905fedce7d59f01d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.434071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.438701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.438941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.458475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1239/2465] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:22.461466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.487178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fbfc7-620f-4929-a57d-083bdcde0392 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.508006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.509925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.510431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.510967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.517908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.526072 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.539956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee22783b3234c4f905fedce7d59f01d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:22.542444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1240/2466] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:22.548024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.562337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee22783b3234c4f905fedce7d59f01d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:22.563711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee22783b3234c4f905fedce7d59f01d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:22.564784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f335504-da58-4c58-a47b-17382df50547 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.566056 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=79,keystone:DELETE=15,keystone:INSERT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:22.572699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.573549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.573918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.574223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.575450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.579101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.579827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee22783b3234c4f905fedce7d59f01d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:22.591049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1241/2467] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:22.597376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.601396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:22.614654 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd4dba0e-c754-44d7-96cc-b514dbd552e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee22783b3234c4f905fedce7d59f01d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:22.620368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a22b1-3c21-4f95-b443-8c20261b6aba admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.625648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1227/2468] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:22 2022] DELETE /identity/v3/users/1ee22783b3234c4f905fedce7d59f01d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:22.724941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:22.744716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.745736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.746096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.746454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] PATH_INFO: `/v3/projects/084ba7885845429c9c566289d342cd7c/users/6813baa9d1d94ed4a92cd206e7d1e820/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.774830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=084ba7885845429c9c566289d342cd7c, user_id=6813baa9d1d94ed4a92cd206e7d1e820, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.778676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.802413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1242/2469] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:22 2022] PUT /identity/v3/projects/084ba7885845429c9c566289d342cd7c/users/6813baa9d1d94ed4a92cd206e7d1e820/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:22.824828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.825302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.825957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.876974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] MFA Rules not processed for user `6813baa9d1d94ed4a92cd206e7d1e820`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:22.880162 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:22.998009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.000827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:23.017420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1228/2470] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:23.018561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.020231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:23.111713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f099a-eb9c-4754-be78-7c9f1d676c25 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.125799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.126616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.126907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.127217 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] PATH_INFO: `/v3/projects/d54636d00282483381454b9be56a43b5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.133340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54636d00282483381454b9be56a43b5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.136259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.150513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6a788-adb4-41c2-b24b-ea64e626e7e1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.156664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cc902-6738-4df5-990d-8d7271163a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.156920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cc902-6738-4df5-990d-8d7271163a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.157179 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cc902-6738-4df5-990d-8d7271163a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.157427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cc902-6738-4df5-990d-8d7271163a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.168602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cc902-6738-4df5-990d-8d7271163a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.171032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cc902-6738-4df5-990d-8d7271163a5a service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.186619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1229/2471] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:39:23.225350 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54636d00282483381454b9be56a43b5', 'request_id': 'req-fa0c49b9-b592-4279-ac15-f1faeabc0305'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:23.243851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54636d00282483381454b9be56a43b5', 'request_id': 'req-fa0c49b9-b592-4279-ac15-f1faeabc0305'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:23.250964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54636d00282483381454b9be56a43b5', 'request_id': 'req-fa0c49b9-b592-4279-ac15-f1faeabc0305'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:23.273679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] The token cache is being invalidate because project d54636d00282483381454b9be56a43b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:23.273679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:23.276418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fa0c49b9-b592-4279-ac15-f1faeabc0305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:23.280511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1243/2472] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:22 2022] DELETE /identity/v3/projects/d54636d00282483381454b9be56a43b5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:23.291818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.292447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.293000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.339847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:23.345398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:23.433009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1230/2473] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:23.441979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.442305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.442788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.470869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:23.472218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:23.481107 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1244/2474] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:23.486383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.494125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:23.500347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1f80-dd54-4520-9bbe-5191dea8c6a1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.508254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.509061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.509439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.509795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.510617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:23.510998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.512829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.530348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1231/2475] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:39:23.534807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.544512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee696b1-6a01-4656-9f8d-67218381050f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.550157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.550831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.551148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.551426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.552145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.554077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.586979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1245/2476] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:23 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:39:23.591272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.602914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275e14-7965-42f2-8aec-350bd57d3c96 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.671173 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.671841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.672137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.672398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.673094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.675089 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.698734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1232/2477] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:23 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:39:23.704741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.714677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb631a3-4c35-4f2e-a96e-4d1e104929f1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.720225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.721123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.721372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.721641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.722399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.724191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.736094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1246/2478] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:23.740394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.751284 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e1094-64fd-4d91-83f7-a552ad991ac1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.761900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee20d6-dd19-4320-ab5c-35ec5bc81b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.762670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee20d6-dd19-4320-ab5c-35ec5bc81b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.763033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee20d6-dd19-4320-ab5c-35ec5bc81b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.763418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee20d6-dd19-4320-ab5c-35ec5bc81b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.764908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee20d6-dd19-4320-ab5c-35ec5bc81b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.767370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee20d6-dd19-4320-ab5c-35ec5bc81b14 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.778507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1233/2479] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:39:23.783259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:23.793228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a30884-7899-4460-80b9-b3d959996204 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:23.798511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:23.799436 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.799719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.799993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] PATH_INFO: `/v3/projects/c548953e0659463ca828aa2958ac7ef4/users/3e1b4f2520734221800c2e4283550346/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.815980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c548953e0659463ca828aa2958ac7ef4, user_id=3e1b4f2520734221800c2e4283550346, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:23.818717 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:23.821006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:23.835580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1247/2480] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:23 2022] PUT /identity/v3/projects/c548953e0659463ca828aa2958ac7ef4/users/3e1b4f2520734221800c2e4283550346/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:23.846319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:23.846640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:23.846927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:23.874167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] MFA Rules not processed for user `3e1b4f2520734221800c2e4283550346`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:23.875476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:23.949016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1234/2481] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:23.954232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.018493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c66ad-e296-43b4-ad64-c8f2b5ae11bd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.023232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.023896 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.024182 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.024486 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.025197 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.027435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.054192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1248/2482] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:23 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:24.058278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.069701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773dd2e-cab6-44e1-9c0e-1a6011975280 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.128045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.128733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.129020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.129299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.130034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.131990 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.138790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:24.154068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1235/2483] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:24 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:24.158218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.169130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb756a03-d63d-491c-b5dc-1b03df1023dd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.174372 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.175033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.175348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.175625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.176349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.177956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.189103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1249/2484] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:24.193399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.203947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145fa9b-54b7-46d6-916e-57b11ac319da admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.208523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.209409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.209691 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.209969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] PATH_INFO: `/v3/projects/adfd6efc7cac455ab7e09fcde7872147/users/7a9abf0f67d543a2ba90492fc53aa9b8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.234967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfd6efc7cac455ab7e09fcde7872147, user_id=7a9abf0f67d543a2ba90492fc53aa9b8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.237672 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.253281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1236/2485] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:24 2022] PUT /identity/v3/projects/adfd6efc7cac455ab7e09fcde7872147/users/7a9abf0f67d543a2ba90492fc53aa9b8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:24.257311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.268172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a12a3a-4a84-47cc-9946-d9324bab09a6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.332459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781144a-6ed7-4760-91de-a960493b8627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.333136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781144a-6ed7-4760-91de-a960493b8627 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.333422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781144a-6ed7-4760-91de-a960493b8627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.333696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781144a-6ed7-4760-91de-a960493b8627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.334435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781144a-6ed7-4760-91de-a960493b8627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.336105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781144a-6ed7-4760-91de-a960493b8627 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.347523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1250/2486] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:24.352322 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.364125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899585e-3ff8-4826-8160-9e44be191033 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.368914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.369834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.370119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.370395 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] PATH_INFO: `/v3/projects/adfd6efc7cac455ab7e09fcde7872147/users/7a9abf0f67d543a2ba90492fc53aa9b8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.372500 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfd6efc7cac455ab7e09fcde7872147, user_id=7a9abf0f67d543a2ba90492fc53aa9b8, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.375471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.386285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1237/2487] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:24 2022] PUT /identity/v3/projects/adfd6efc7cac455ab7e09fcde7872147/users/7a9abf0f67d543a2ba90492fc53aa9b8/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:24.398099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.398397 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.398792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.427824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] MFA Rules not processed for user `7a9abf0f67d543a2ba90492fc53aa9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:24.429054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:24.483175 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:24.503352 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1251/2488] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:24.517363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.519229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:24.582578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa8f6-1ec5-4e81-8292-29c911e41630 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.586512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.587246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.587533 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.587814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.598109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.599623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.610939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1238/2489] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:24.694561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.705884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5adb8e-feec-4149-a4cd-b3ca90ee88c5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.709704 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd189a2-b2f3-40db-86ed-1fb2715e808a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.710435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd189a2-b2f3-40db-86ed-1fb2715e808a service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.710865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd189a2-b2f3-40db-86ed-1fb2715e808a service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.711247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd189a2-b2f3-40db-86ed-1fb2715e808a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.781084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd189a2-b2f3-40db-86ed-1fb2715e808a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.782811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd189a2-b2f3-40db-86ed-1fb2715e808a service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.797394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1252/2490] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:32.590145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:32.591597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:32.657697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7171840-2638-47dd-87e5-d7db581330f7 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:32.662255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:32.663153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:32.663516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:32.663696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] PATH_INFO: `/v3/users/6813baa9d1d94ed4a92cd206e7d1e820` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:32.664884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6813baa9d1d94ed4a92cd206e7d1e820)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:32.666812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:32.713193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6813baa9d1d94ed4a92cd206e7d1e820'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:32.721228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6813baa9d1d94ed4a92cd206e7d1e820'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:32.721863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6813baa9d1d94ed4a92cd206e7d1e820'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:32.724000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:32.730001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6813baa9d1d94ed4a92cd206e7d1e820'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:32.743652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6813baa9d1d94ed4a92cd206e7d1e820'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:32.750003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1239/2491] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:32 2022] DELETE /identity/v3/users/6813baa9d1d94ed4a92cd206e7d1e820 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:32.754619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8d95-0781-47b0-bc1d-c68640a9cc23 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:32.755113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8d95-0781-47b0-bc1d-c68640a9cc23 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:32.755212 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8d95-0781-47b0-bc1d-c68640a9cc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:32.772623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-adff8d95-0781-47b0-bc1d-c68640a9cc23 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:32.773831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-adff8d95-0781-47b0-bc1d-c68640a9cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:32.834836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1253/2492] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:32.845199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:32.846606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:32.908980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8638-4871-42e0-95a4-005c72cd5866 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:32.912641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:32.913302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:32.913588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:32.913867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:32.923605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:32.925053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:32.934803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1240/2493] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:33.096292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:33.096605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:33.097042 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:33.115678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:33.116884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:33.176347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1254/2494] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jul 21 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:33.188966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:33.190493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:33.219035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:33.220580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:33.237449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1522a-3125-446c-ad13-974a5a1687fe None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:33.242398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:33.243407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:33.243768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:33.244127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] PATH_INFO: `/v3/projects/084ba7885845429c9c566289d342cd7c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:33.245538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=084ba7885845429c9c566289d342cd7c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:33.247961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:33.270387 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a418be6-1db5-4cc6-8e55-110e1445f6da service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:33.275506 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:33.276167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:33.276451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:33.276731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:33.287324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:33.290442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:33.292487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:33.306226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1241/2495] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:39:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 22:39:33.316478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084ba7885845429c9c566289d342cd7c', 'request_id': 'req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:33.323041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084ba7885845429c9c566289d342cd7c', 'request_id': 'req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:33.336875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084ba7885845429c9c566289d342cd7c', 'request_id': 'req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:33.354877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] The token cache is being invalidate because project 084ba7885845429c9c566289d342cd7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:33.355155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:33.355829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3ea572ba-d93e-441b-ae90-e51c3fb27bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:33.357476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1255/2496] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:33 2022] DELETE /identity/v3/projects/084ba7885845429c9c566289d342cd7c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:34.855355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:34.856774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:34.870688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2299f40-699a-41b0-94a4-944ca61e2f18 service swift] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:34.932827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:34.933635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:34.933921 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:34.934196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] PATH_INFO: `/v3/users/3e1b4f2520734221800c2e4283550346` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:34.935455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1b4f2520734221800c2e4283550346)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:34.937948 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:34.943892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:34.985524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1b4f2520734221800c2e4283550346'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:34.992623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1b4f2520734221800c2e4283550346'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:34.993272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1b4f2520734221800c2e4283550346'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.001403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1b4f2520734221800c2e4283550346'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.013422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1b4f2520734221800c2e4283550346'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.020484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1242/2497] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:34 2022] DELETE /identity/v3/users/3e1b4f2520734221800c2e4283550346 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:35.025189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:35.025672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:35.025770 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:35.043438 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:35.044655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:35.105395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1256/2498] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:35.115469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:35.116865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:35.172954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5008f8a-9b05-4670-9642-f63320d204f5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:35.176646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:35.177288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:35.177564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:35.177838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:35.187551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:35.189034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:35.198511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1243/2499] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:35.470361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:35.471904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:35.484782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1baea-9548-4169-bfc3-b04b7e34e558 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:35.489610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:35.490476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:35.490786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:35.491665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] PATH_INFO: `/v3/projects/c548953e0659463ca828aa2958ac7ef4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:35.492539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c548953e0659463ca828aa2958ac7ef4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:35.494514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:35.547274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c548953e0659463ca828aa2958ac7ef4', 'request_id': 'req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.554049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c548953e0659463ca828aa2958ac7ef4', 'request_id': 'req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.559823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c548953e0659463ca828aa2958ac7ef4', 'request_id': 'req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.573670 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] The token cache is being invalidate because project c548953e0659463ca828aa2958ac7ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:35.573957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.574572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.576147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1257/2500] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:35 2022] DELETE /identity/v3/projects/c548953e0659463ca828aa2958ac7ef4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:35.581127 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:35.591392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff0d0f-e4fe-4772-9850-06d64353c07d service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:35.647705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:35.648583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:35.648868 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:35.649141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] PATH_INFO: `/v3/users/7a9abf0f67d543a2ba90492fc53aa9b8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:35.650377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a9abf0f67d543a2ba90492fc53aa9b8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:35.652337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:35.686144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:35.704734 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a9abf0f67d543a2ba90492fc53aa9b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.712503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a9abf0f67d543a2ba90492fc53aa9b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.713437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a9abf0f67d543a2ba90492fc53aa9b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.720826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a9abf0f67d543a2ba90492fc53aa9b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.733904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5cc0907f-2faf-499d-93b1-d2839f1ad2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a9abf0f67d543a2ba90492fc53aa9b8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:35.742765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1244/2501] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:35 2022] DELETE /identity/v3/users/7a9abf0f67d543a2ba90492fc53aa9b8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:36.013021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.014473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:36.028200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c9108-c487-4e04-8c59-56fb6ce9a6d3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.069242 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:36.089273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.090141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.090422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.090695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] PATH_INFO: `/v3/projects/adfd6efc7cac455ab7e09fcde7872147` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.091913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adfd6efc7cac455ab7e09fcde7872147)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.093834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.142255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adfd6efc7cac455ab7e09fcde7872147', 'request_id': 'req-53531cd9-2711-4df7-8bf3-e2bec779bfc5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:36.148286 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adfd6efc7cac455ab7e09fcde7872147', 'request_id': 'req-53531cd9-2711-4df7-8bf3-e2bec779bfc5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:36.153657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adfd6efc7cac455ab7e09fcde7872147', 'request_id': 'req-53531cd9-2711-4df7-8bf3-e2bec779bfc5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:36.166597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] The token cache is being invalidate because project adfd6efc7cac455ab7e09fcde7872147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:36.166915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:36.167555 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-53531cd9-2711-4df7-8bf3-e2bec779bfc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:36.169096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1258/2502] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:36 2022] DELETE /identity/v3/projects/adfd6efc7cac455ab7e09fcde7872147 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:36.175792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.176321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.176422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.194807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:36.196051 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:36.254261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1245/2503] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:36.262185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.262415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.262692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.282713 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:36.283979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:36.299941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1259/2504] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:36.304262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.314409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019bf00-eefc-4717-8a68-c0276fcfae90 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.319142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.319801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.320085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.320360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.321090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:36.321401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.323011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.334918 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1246/2505] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:36.339458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.351224 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fbcf7-7416-44fe-b241-d3e9659e4c71 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.356097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.356761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.357048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.357328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.358025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.360595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.385783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1260/2506] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:36.390355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.400390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080af5-e3e2-4f3a-8c41-80c7fd4bd473 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.457296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.457971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.458257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.458543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.459261 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.461194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.481904 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1247/2507] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:39:36.487278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.497758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d6d5d-d610-4d39-9676-582c0a590065 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.502692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.503407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.503688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.503963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.504687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.506297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.517531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1261/2508] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:36.521578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.532103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3ecf-31e0-4650-b511-5049826744ce admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.536739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd081ece-b769-4769-8fe3-16584b194069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.537392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd081ece-b769-4769-8fe3-16584b194069 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.537673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd081ece-b769-4769-8fe3-16584b194069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.537947 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd081ece-b769-4769-8fe3-16584b194069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.538660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd081ece-b769-4769-8fe3-16584b194069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.540290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd081ece-b769-4769-8fe3-16584b194069 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.550725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1248/2509] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:36.554804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.565351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3d852-2d08-4892-82c7-703c43a347b2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.569929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.570823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.571144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.571427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] PATH_INFO: `/v3/projects/1567f7b34a6a49ea8f49ca3048ebe0b3/users/cc8a750496c148a5a5ff89553e1e9727/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.587566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1567f7b34a6a49ea8f49ca3048ebe0b3, user_id=cc8a750496c148a5a5ff89553e1e9727, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.590249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.605431 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1262/2510] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:36 2022] PUT /identity/v3/projects/1567f7b34a6a49ea8f49ca3048ebe0b3/users/cc8a750496c148a5a5ff89553e1e9727/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:36.616119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.616408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.616741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.642451 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:36.646058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] MFA Rules not processed for user `cc8a750496c148a5a5ff89553e1e9727`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:36.648423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:36.716483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1249/2511] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:36.722190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.786961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b20748-81e5-4076-a0b3-c5efd655388c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.791819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.792483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.792768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.793046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.793784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.795766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.806246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:36.823852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1263/2512] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:39:36.828042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.838812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e2a30-c314-4ea9-876a-4b3437d6d3a7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.896702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.897370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.897657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.897930 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.898624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.900586 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.920326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1250/2513] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:36 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:36.924516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.935306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837afe17-4f4a-43f1-b3ca-daed3fbd4a7e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.940904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.941570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.941861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.942145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.942876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.944546 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:36.955907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1264/2514] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:36.960057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:36.969923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7f62a-1b7e-4126-ba6c-b2c6acb806d8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:36.974526 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:36.975437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:36.975715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:36.975988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] PATH_INFO: `/v3/projects/2ca0ca2af9f546f4ac95714b2db9e114/users/3d67c915ee924d80996700c5b37ed71b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:36.994301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca0ca2af9f546f4ac95714b2db9e114, user_id=3d67c915ee924d80996700c5b37ed71b, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:36.997019 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:37.012864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1251/2515] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:36 2022] PUT /identity/v3/projects/2ca0ca2af9f546f4ac95714b2db9e114/users/3d67c915ee924d80996700c5b37ed71b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:37.017425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:37.027232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee12cac-9996-4f3c-b870-86c8c795619c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:37.084933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fe055-8beb-4003-996f-588d9ff67442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:37.085614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe055-8beb-4003-996f-588d9ff67442 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.085805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe055-8beb-4003-996f-588d9ff67442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.086077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fe055-8beb-4003-996f-588d9ff67442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.086795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fe055-8beb-4003-996f-588d9ff67442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:37.088471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fe055-8beb-4003-996f-588d9ff67442 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:37.098691 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1265/2516] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:37.103367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:37.113899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616ef03-6d2b-457b-b8da-ca3e73b323df admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:37.118623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:37.119552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.119836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.120117 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] PATH_INFO: `/v3/projects/2ca0ca2af9f546f4ac95714b2db9e114/users/3d67c915ee924d80996700c5b37ed71b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.122118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca0ca2af9f546f4ac95714b2db9e114, user_id=3d67c915ee924d80996700c5b37ed71b, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:37.125380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:37.136403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1252/2517] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:37 2022] PUT /identity/v3/projects/2ca0ca2af9f546f4ac95714b2db9e114/users/3d67c915ee924d80996700c5b37ed71b/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:37.148461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.148750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.149056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.175083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] MFA Rules not processed for user `3d67c915ee924d80996700c5b37ed71b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:37.176390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:37.244006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1266/2518] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:37.256177 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:37.257962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:37.315288 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d5864-19d3-48de-b452-404b888eaacc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:37.318942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:37.319620 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.319908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.320187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.329367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:37.330807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:37.334753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:37.342587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1253/2519] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:37.429876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:37.440598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e04ad-d661-49f4-9df5-9ecbdb603533 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:37.444391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d715dd-ca77-4ec7-a456-950f41e8489f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:37.445073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d715dd-ca77-4ec7-a456-950f41e8489f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.445361 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d715dd-ca77-4ec7-a456-950f41e8489f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.445671 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d715dd-ca77-4ec7-a456-950f41e8489f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.505195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d715dd-ca77-4ec7-a456-950f41e8489f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:37.506668 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d715dd-ca77-4ec7-a456-950f41e8489f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:37.511240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:37.516590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1267/2520] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:37.785495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:37.853808 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae59f1-636c-43ed-a7ce-4454acc3c919 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:37.859011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:37.859905 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.860309 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.860592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] PATH_INFO: `/v3/users/cc8a750496c148a5a5ff89553e1e9727` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.861820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8a750496c148a5a5ff89553e1e9727)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:37.863803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:37.910392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8a750496c148a5a5ff89553e1e9727'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:37.917310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8a750496c148a5a5ff89553e1e9727'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:37.917920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8a750496c148a5a5ff89553e1e9727'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:37.925687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8a750496c148a5a5ff89553e1e9727'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:37.937328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8a750496c148a5a5ff89553e1e9727'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:37.943189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1254/2521] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:37 2022] DELETE /identity/v3/users/cc8a750496c148a5a5ff89553e1e9727 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:37.947911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2275edf-bf20-494c-9728-145628737d1f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:37.948211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2275edf-bf20-494c-9728-145628737d1f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:37.948519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2275edf-bf20-494c-9728-145628737d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:37.965788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f2275edf-bf20-494c-9728-145628737d1f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:37.966994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f2275edf-bf20-494c-9728-145628737d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:38.028041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1268/2522] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:38.037804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:38.039253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:38.096304 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43fa6d-403a-495e-872f-70b7d2bad563 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:38.099922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:38.100571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:38.100855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:38.101128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:38.111323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:38.112743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:38.122401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1255/2523] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:38.404060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2275edf-bf20-494c-9728-145628737d1f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:38.405621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f2275edf-bf20-494c-9728-145628737d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:38.417659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2275edf-bf20-494c-9728-145628737d1f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:38.422405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:38.423394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:38.423553 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:38.423831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] PATH_INFO: `/v3/projects/1567f7b34a6a49ea8f49ca3048ebe0b3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:38.425095 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1567f7b34a6a49ea8f49ca3048ebe0b3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:38.427044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:38.598838 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1567f7b34a6a49ea8f49ca3048ebe0b3', 'request_id': 'req-f4ecef98-0db3-4e83-ae76-1831ff5655b5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.605228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1567f7b34a6a49ea8f49ca3048ebe0b3', 'request_id': 'req-f4ecef98-0db3-4e83-ae76-1831ff5655b5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.611178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1567f7b34a6a49ea8f49ca3048ebe0b3', 'request_id': 'req-f4ecef98-0db3-4e83-ae76-1831ff5655b5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.623601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] The token cache is being invalidate because project 1567f7b34a6a49ea8f49ca3048ebe0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:38.623894 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.624549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.626788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1269/2524] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:38 2022] DELETE /identity/v3/projects/1567f7b34a6a49ea8f49ca3048ebe0b3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:38.630889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:38.640708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12692d43-1f76-4d77-894c-b6f0f4177255 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:38.657679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:38.702203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:38.703032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:38.703343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:38.703754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] PATH_INFO: `/v3/users/3d67c915ee924d80996700c5b37ed71b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:38.705937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d67c915ee924d80996700c5b37ed71b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:38.707932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:38.755940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d67c915ee924d80996700c5b37ed71b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.762937 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d67c915ee924d80996700c5b37ed71b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.763664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d67c915ee924d80996700c5b37ed71b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.771638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d67c915ee924d80996700c5b37ed71b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.783325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b9d4d0e2-62bb-4854-8446-0b0ea16b8ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d67c915ee924d80996700c5b37ed71b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:38.789203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1256/2525] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:38 2022] DELETE /identity/v3/users/3d67c915ee924d80996700c5b37ed71b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:39.079971 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.081531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:39.097070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecef98-0db3-4e83-ae76-1831ff5655b5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.156609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.158449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.158727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.159002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] PATH_INFO: `/v3/projects/2ca0ca2af9f546f4ac95714b2db9e114` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.160238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca0ca2af9f546f4ac95714b2db9e114)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.162159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.194484 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:39.212288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca0ca2af9f546f4ac95714b2db9e114', 'request_id': 'req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:39.218287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca0ca2af9f546f4ac95714b2db9e114', 'request_id': 'req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:39.224437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca0ca2af9f546f4ac95714b2db9e114', 'request_id': 'req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:39.238434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] The token cache is being invalidate because project 2ca0ca2af9f546f4ac95714b2db9e114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:39.238721 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:39.239399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f96825ff-38fd-4484-85e4-0f2b4ba6d1d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:39.240981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1270/2526] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:39 2022] DELETE /identity/v3/projects/2ca0ca2af9f546f4ac95714b2db9e114 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:39.246145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.246434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.246780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.264564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:39.266053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:39.325352 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1257/2527] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:39.333193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.333481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.333763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.353779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:39.354988 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:39.363453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1271/2528] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:39.367899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.377789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f23880-6a6a-47d2-a879-f07fad9ca7c5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.382379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.383047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.383357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.383638 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.384372 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:39.384681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.386292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.398277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1258/2529] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:39.402645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.412513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80edc0-3a07-402c-ae91-7d2ed14b9b1f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.417231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.417881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.418162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.418442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.419293 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.421196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.447768 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1272/2530] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:39 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:39.451963 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.462039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad167ef-3de0-4600-8238-463476ccbb10 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.487488 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:39.520992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.521657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.521940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.522211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.522898 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.524829 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.545272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1259/2531] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:39 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:39.549889 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.560142 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e76e-0c4a-45f4-a349-b2993a3d26de admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.564781 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.565439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.565725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.566003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.566762 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.568406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.579109 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1273/2532] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:39.583165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.592984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791c3f-1aa7-4a88-b1b9-f871059cdf04 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.598763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc4c98-ee10-42cf-8862-1d5fbf2f9352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.599529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4c98-ee10-42cf-8862-1d5fbf2f9352 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.599849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4c98-ee10-42cf-8862-1d5fbf2f9352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.600163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4c98-ee10-42cf-8862-1d5fbf2f9352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.601055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc4c98-ee10-42cf-8862-1d5fbf2f9352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.602693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc4c98-ee10-42cf-8862-1d5fbf2f9352 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.613048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1260/2533] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:39.617123 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.627040 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6204125-1718-45d5-9e52-72c8c286994e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.631657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.632542 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.632828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.633103 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] PATH_INFO: `/v3/projects/017f6e12d4f2484487673b9718ec147c/users/f32398a938b644e09fb2bbf28dce40fd/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.648632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017f6e12d4f2484487673b9718ec147c, user_id=f32398a938b644e09fb2bbf28dce40fd, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.651324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.665125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1274/2534] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:39 2022] PUT /identity/v3/projects/017f6e12d4f2484487673b9718ec147c/users/f32398a938b644e09fb2bbf28dce40fd/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:39.670293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff136-73ed-482d-b98b-5fe09ae6cf18 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.670726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff136-73ed-482d-b98b-5fe09ae6cf18 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.670836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff136-73ed-482d-b98b-5fe09ae6cf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.689210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-60cff136-73ed-482d-b98b-5fe09ae6cf18 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:39.690419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-60cff136-73ed-482d-b98b-5fe09ae6cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:39.752511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1261/2535] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:39.762533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:39.764421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:39.821170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a428f72-9d41-468b-817f-8002a3ad1ca7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:39.824805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:39.825454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:39.825741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:39.826021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:39.835550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:39.836996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:39.847080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1275/2536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:40.482788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.483117 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.483406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.512655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] MFA Rules not processed for user `f32398a938b644e09fb2bbf28dce40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:40.513889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:40.587340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1262/2537] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:40.591932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:40.602204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f661f-8e7b-41ef-ae42-8891b8bc2721 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:40.606976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:40.607593 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.607876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.608150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.608862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:40.610802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:40.636632 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1276/2538] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:40 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:40.641494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:40.653112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92537c-75d4-426e-9425-0b5e2dc04ef7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:40.694059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:40.725330 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:40.726142 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.726487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.726827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.727659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:40.729750 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:40.752591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1263/2539] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:40 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:40.756855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:40.768134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b98756-6e5c-458d-8284-396eb76c198e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:40.772984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:40.773672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.773959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.774240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.775012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:40.776678 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:40.789656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1277/2540] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:39:40.794087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:40.806312 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8d87-d32a-42b0-bcf1-a8adc8757d41 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:40.816673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:40.817889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.818294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.818726 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] PATH_INFO: `/v3/projects/47ad890023104d4ead28aeba70ab0809/users/8b74b26f8885496ca1da3d4d2d627469/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.835761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ad890023104d4ead28aeba70ab0809, user_id=8b74b26f8885496ca1da3d4d2d627469, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:40.838499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:40.854367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1264/2541] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:40 2022] PUT /identity/v3/projects/47ad890023104d4ead28aeba70ab0809/users/8b74b26f8885496ca1da3d4d2d627469/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:40.858494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:40.868922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a963-4640-4943-a095-09fd0d81f417 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:40.884840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:40.928537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734f09d-0f16-40ea-9534-bf8620cf83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:40.929215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734f09d-0f16-40ea-9534-bf8620cf83a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.929502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734f09d-0f16-40ea-9534-bf8620cf83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.929780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734f09d-0f16-40ea-9534-bf8620cf83a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.930514 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734f09d-0f16-40ea-9534-bf8620cf83a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:40.932172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734f09d-0f16-40ea-9534-bf8620cf83a5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:40.943338 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1278/2542] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:39:40.947535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:40.958941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9735b99-285a-430c-a015-2934b891ef47 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:40.963979 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:40.964877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:40.966081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:40.966081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] PATH_INFO: `/v3/projects/47ad890023104d4ead28aeba70ab0809/users/8b74b26f8885496ca1da3d4d2d627469/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:40.967469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ad890023104d4ead28aeba70ab0809, user_id=8b74b26f8885496ca1da3d4d2d627469, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:40.971215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:40.982079 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1265/2543] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:40 2022] PUT /identity/v3/projects/47ad890023104d4ead28aeba70ab0809/users/8b74b26f8885496ca1da3d4d2d627469/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:41.601145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:41.601700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:41.601835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:41.628797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] MFA Rules not processed for user `8b74b26f8885496ca1da3d4d2d627469`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:41.630000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:41.706116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1279/2544] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:41 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:41.717471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:41.719772 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:41.780524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085675b-356f-445a-acc0-e987a5c6bbf0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:41.784255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:41.784910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:41.785195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:41.785478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:41.841099 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:41.842758 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:41.845736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:41.847484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:41.857610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1266/2545] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:41.926455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43122849-cd45-488d-98ea-17eb41a272c7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:41.932179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:41.932857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:41.933140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:41.933413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:41.935259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:41.976031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:41.999944 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:42.002180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:42.018529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1280/2546] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:39:43.237919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:43.246603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746ddc-5051-460b-b7c6-7c05b7c0d724 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:43.250316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:43.250946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:43.251265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:43.251541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:43.260666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:43.262081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:43.272792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1267/2547] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:52.008515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=10 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:53.268786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:56.458425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:56.522558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac84b7-f0e5-4498-8165-4bd9fb8ab8c1 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:56.526904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:56.527807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:56.528085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:56.528353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] PATH_INFO: `/v3/users/f32398a938b644e09fb2bbf28dce40fd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:56.529522 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32398a938b644e09fb2bbf28dce40fd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:56.531478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:56.576736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32398a938b644e09fb2bbf28dce40fd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.584552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32398a938b644e09fb2bbf28dce40fd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.596631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32398a938b644e09fb2bbf28dce40fd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.597209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32398a938b644e09fb2bbf28dce40fd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.604106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32398a938b644e09fb2bbf28dce40fd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.610035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1281/2548] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:56 2022] DELETE /identity/v3/users/f32398a938b644e09fb2bbf28dce40fd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:56.795985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:56.807861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d30bac-9aac-41c4-a1c7-a27d9c7bae42 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:56.868034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:56.868899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:56.869182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:56.869460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] PATH_INFO: `/v3/projects/017f6e12d4f2484487673b9718ec147c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:56.870640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017f6e12d4f2484487673b9718ec147c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:56.872590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:56.920528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017f6e12d4f2484487673b9718ec147c', 'request_id': 'req-4147a57d-0a69-4e50-b585-4178e5b0d0d3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.928450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017f6e12d4f2484487673b9718ec147c', 'request_id': 'req-4147a57d-0a69-4e50-b585-4178e5b0d0d3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.934865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017f6e12d4f2484487673b9718ec147c', 'request_id': 'req-4147a57d-0a69-4e50-b585-4178e5b0d0d3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.945842 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] The token cache is being invalidate because project 017f6e12d4f2484487673b9718ec147c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:56.946137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.946813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:56.948383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1268/2549] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:56 2022] DELETE /identity/v3/projects/017f6e12d4f2484487673b9718ec147c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:56.952406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:56.953849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:56.967440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd255ffa-68ac-428c-943b-0575238c4d30 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.025310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.026176 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.026458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.026733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] PATH_INFO: `/v3/users/8b74b26f8885496ca1da3d4d2d627469` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.027957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b74b26f8885496ca1da3d4d2d627469)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.029874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.052078 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:57.078027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b74b26f8885496ca1da3d4d2d627469'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.085911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b74b26f8885496ca1da3d4d2d627469'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.102926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b74b26f8885496ca1da3d4d2d627469'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.104334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b74b26f8885496ca1da3d4d2d627469'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.116985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-96228e1a-a1a4-4147-a933-43f1c68d6e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b74b26f8885496ca1da3d4d2d627469'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.127060 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1282/2550] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:39:56 2022] DELETE /identity/v3/users/8b74b26f8885496ca1da3d4d2d627469 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:57.313445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.315557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:57.328886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147a57d-0a69-4e50-b585-4178e5b0d0d3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.391013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.391977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.392265 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.392546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] PATH_INFO: `/v3/projects/47ad890023104d4ead28aeba70ab0809` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.393894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ad890023104d4ead28aeba70ab0809)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.396379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.433983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:57.449547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ad890023104d4ead28aeba70ab0809', 'request_id': 'req-323dc3df-98ec-4d9e-94c1-f99001057d5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.457712 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ad890023104d4ead28aeba70ab0809', 'request_id': 'req-323dc3df-98ec-4d9e-94c1-f99001057d5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.464262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ad890023104d4ead28aeba70ab0809', 'request_id': 'req-323dc3df-98ec-4d9e-94c1-f99001057d5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.474977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] The token cache is being invalidate because project 47ad890023104d4ead28aeba70ab0809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:39:57.475296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.476363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-323dc3df-98ec-4d9e-94c1-f99001057d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:39:57.477957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1269/2551] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:39:57 2022] DELETE /identity/v3/projects/47ad890023104d4ead28aeba70ab0809 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:39:57.482962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.483450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.483587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.501853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:57.503711 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:57.563158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1283/2552] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:57.571597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.571886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.572193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.590693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:57.591931 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:57.600144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1270/2553] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:57.604461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.615185 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe3b4c-0be7-4584-8dab-ecfef3a23711 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.620799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.621461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.621746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.622025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.622750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:57.623083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.624696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.637065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1284/2554] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:39:57.641717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.651239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23fdfa-bcb0-40b8-ba77-a1a68c12c6ea None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.656076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.656737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.657022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.657296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.658290 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.660048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.687786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1271/2555] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:39:57 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:57.693582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.705620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f7c25-6ccd-4ed1-b406-995d1b2f5909 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.768882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.769563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.769853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.770138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.770849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.772832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.794577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1285/2556] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:39:57 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:57.799470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.811521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a634e-ffec-499a-9d12-7bed8ea2a71d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.818576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.819223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.819508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.819780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.820512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.822116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.833433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1272/2557] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:57.837473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.848303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866cbae-a099-4f01-a20c-74cd16a054e2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.853108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a361a5-80a9-4b62-980c-378b4d16c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.853774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a361a5-80a9-4b62-980c-378b4d16c461 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.854084 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a361a5-80a9-4b62-980c-378b4d16c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.854391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a361a5-80a9-4b62-980c-378b4d16c461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.855160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a361a5-80a9-4b62-980c-378b4d16c461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.856774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a361a5-80a9-4b62-980c-378b4d16c461 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.868473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1286/2558] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:57.872907 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:57.883543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885597b3-df83-4e3d-a94a-8905e56854f1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:57.888278 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:57.889163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.889697 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.889982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] PATH_INFO: `/v3/projects/1d10b882ad80447aa37649632a0d5692/users/d84260bc6c714b7da93086841746a53a/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.904836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d10b882ad80447aa37649632a0d5692, user_id=d84260bc6c714b7da93086841746a53a, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:57.907744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:57.921953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1273/2559] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:39:57 2022] PUT /identity/v3/projects/1d10b882ad80447aa37649632a0d5692/users/d84260bc6c714b7da93086841746a53a/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:39:57.928027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eba3b-37e1-47df-8f81-16ab2ff107ba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:57.928319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eba3b-37e1-47df-8f81-16ab2ff107ba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:57.928610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eba3b-37e1-47df-8f81-16ab2ff107ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:57.935448 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:57.947226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-889eba3b-37e1-47df-8f81-16ab2ff107ba None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:57.948447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-889eba3b-37e1-47df-8f81-16ab2ff107ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:58.009044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1287/2560] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:58.019035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:58.020771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:58.078433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f1d5-aea5-44fc-9eec-b615e033c759 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:58.082053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:58.082698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:58.082977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:58.083279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:58.092397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:58.093815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:58.104814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1274/2561] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:58.767179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:58.768419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:58.768535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:58.796181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] MFA Rules not processed for user `d84260bc6c714b7da93086841746a53a`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:58.797526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:58.876674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1288/2562] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:39:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:39:58.888400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:58.890190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:58.950061 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526470f-4576-49f2-a6b7-cdd609bab29f service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:58.953877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba2dd1-5ebb-4eee-9d1f-0f50e1ee671a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:58.954786 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba2dd1-5ebb-4eee-9d1f-0f50e1ee671a service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:58.954935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba2dd1-5ebb-4eee-9d1f-0f50e1ee671a service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:58.955320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba2dd1-5ebb-4eee-9d1f-0f50e1ee671a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:58.965863 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba2dd1-5ebb-4eee-9d1f-0f50e1ee671a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:58.967426 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba2dd1-5ebb-4eee-9d1f-0f50e1ee671a service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:58.976785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1275/2563] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:59.693585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:59.695028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:59.707750 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b678ad-b041-4f28-922a-331b338aa254 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:59.711335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:59.711984 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:59.712270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:59.712548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:59.714333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:59.726835 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:59.728323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:59.737992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1289/2564] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:00.328698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:00.329269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:00.329269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:00.335647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:00.351520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] MFA Rules not processed for user `d84260bc6c714b7da93086841746a53a`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:00.353806 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:00.365966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1276/2565] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:00.616391 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:00.626346 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5ff57-42bd-4da6-ab99-6d5241b365cf service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:00.631187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:00.631863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:00.632149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:00.632421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:00.634255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:00.645981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:00.648130 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:00.658679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1290/2566] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:03.149030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:03.151022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:03.169983 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbf71-ec2a-43de-8b0a-20f942ae0e9d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:03.178773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:03.179647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:03.179928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:03.180203 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] PATH_INFO: `/v3/users/c4284eb173fc49d1b059fdd2f3de2b4d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:03.181438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4284eb173fc49d1b059fdd2f3de2b4d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:03.183425 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:03.244661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4284eb173fc49d1b059fdd2f3de2b4d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.257239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4284eb173fc49d1b059fdd2f3de2b4d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.258117 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4284eb173fc49d1b059fdd2f3de2b4d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.265876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4284eb173fc49d1b059fdd2f3de2b4d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.280308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b00497d8-94d3-4107-9b81-a88d8963bdd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4284eb173fc49d1b059fdd2f3de2b4d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.287277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1277/2567] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:03 2022] DELETE /identity/v3/users/c4284eb173fc49d1b059fdd2f3de2b4d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:03.488994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:03.505446 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b5f81-cb72-42d3-be5e-04c5c31a7d17 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:03.632479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:03.634035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:03.634540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:03.635114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] PATH_INFO: `/v3/projects/d5c8c5bf1b1c4caeab8b5daf28ada57d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:03.637252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5c8c5bf1b1c4caeab8b5daf28ada57d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:03.647086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:03.675239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:03.726540 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5c8c5bf1b1c4caeab8b5daf28ada57d', 'request_id': 'req-57a12192-0c55-4146-a62b-df20a8f9b8cb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.736973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5c8c5bf1b1c4caeab8b5daf28ada57d', 'request_id': 'req-57a12192-0c55-4146-a62b-df20a8f9b8cb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.748134 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5c8c5bf1b1c4caeab8b5daf28ada57d', 'request_id': 'req-57a12192-0c55-4146-a62b-df20a8f9b8cb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.780272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] The token cache is being invalidate because project d5c8c5bf1b1c4caeab8b5daf28ada57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:03.780461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.781128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-57a12192-0c55-4146-a62b-df20a8f9b8cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:03.783147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1291/2568] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:03 2022] DELETE /identity/v3/projects/d5c8c5bf1b1c4caeab8b5daf28ada57d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:03.788458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:03.789398 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:03.789751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:03.833578 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:03.836993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:03.922917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1278/2569] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:03.932272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:03.932589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:03.932834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:03.954749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:03.956203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:03.964313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1292/2570] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:03.968570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:03.979251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc99f6-6e62-4098-9db7-f34fce088443 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:03.984189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:03.985204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:03.985783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:03.986222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:03.987790 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:03.988169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:03.989871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.008020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1279/2571] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:40:04.009362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:04.019981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5403b-e12b-4a22-9b34-a70e741ef378 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:04.025041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:04.025710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.026220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.026335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.026980 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:04.028952 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.059604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1293/2572] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:04 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:40:04.063900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:04.074195 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7238-5083-4850-a329-1fc250d2904b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:04.137627 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:04.147194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:04.148099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.148765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.149336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.150720 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:04.153338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.176695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1280/2573] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:04 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:04.180726 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:04.191544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfca43c-b439-43ff-ae92-52e834d82e47 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:04.197081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:04.197779 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.198061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.198333 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.199069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:04.200871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.212598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1294/2574] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:04.216698 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:04.229077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac0cf9-218d-4f95-b2c7-6b86bc9d1609 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:04.234443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5b209-516e-452d-ba25-6e976af20d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:04.235137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b209-516e-452d-ba25-6e976af20d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.235430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b209-516e-452d-ba25-6e976af20d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.235714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b209-516e-452d-ba25-6e976af20d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.236497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5b209-516e-452d-ba25-6e976af20d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:04.238262 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5b209-516e-452d-ba25-6e976af20d2b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.249781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1281/2575] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:04.254207 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:04.270068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a619f-7ff9-42bb-8951-d1d8e8e1e62c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:04.275288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:04.276272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.276556 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.276832 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] PATH_INFO: `/v3/projects/c6be7ddcf75e41b2ac131be45afe90c1/users/009131140458431681c58cab8b662a5b/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.292763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6be7ddcf75e41b2ac131be45afe90c1, user_id=009131140458431681c58cab8b662a5b, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:04.295449 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.309106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1295/2576] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:04 2022] PUT /identity/v3/projects/c6be7ddcf75e41b2ac131be45afe90c1/users/009131140458431681c58cab8b662a5b/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:04.319258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.319553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.319835 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.347721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] MFA Rules not processed for user `009131140458431681c58cab8b662a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:04.349016 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:04.415355 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1282/2577] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:04.426483 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:04.428226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:04.510701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa815f-160a-4bcb-979a-95c587edee8a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:04.516467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b9af2-24ab-47af-b10e-3f8e24e47495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:04.517120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b9af2-24ab-47af-b10e-3f8e24e47495 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:04.517956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b9af2-24ab-47af-b10e-3f8e24e47495 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:04.518355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b9af2-24ab-47af-b10e-3f8e24e47495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:04.526440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b9af2-24ab-47af-b10e-3f8e24e47495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:04.528220 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b9af2-24ab-47af-b10e-3f8e24e47495 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:04.537788 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1296/2578] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:12.191332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:12.294167 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fff81-2723-4ca0-9aba-7c1d658581f5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:12.299954 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:12.301182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:12.301182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:12.301392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] PATH_INFO: `/v3/users/009131140458431681c58cab8b662a5b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:12.302595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009131140458431681c58cab8b662a5b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:12.304631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:12.366735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009131140458431681c58cab8b662a5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:12.376222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009131140458431681c58cab8b662a5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:12.377664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009131140458431681c58cab8b662a5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:12.385385 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009131140458431681c58cab8b662a5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:12.388105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:12.403388 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009131140458431681c58cab8b662a5b'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:12.409871 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1283/2579] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:12 2022] DELETE /identity/v3/users/009131140458431681c58cab8b662a5b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:12.414696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:12.415180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:12.415277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:12.443398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:12.444960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:12.459283 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:12.520377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1297/2580] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:12.529850 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:12.531428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:12.596932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09bfff-0d31-4af9-adb4-ab2780e950e9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:12.600708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609875c-1b91-4bb3-8846-83ec8cae444d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:12.601369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609875c-1b91-4bb3-8846-83ec8cae444d service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:12.601655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609875c-1b91-4bb3-8846-83ec8cae444d service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:12.601934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609875c-1b91-4bb3-8846-83ec8cae444d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:12.612329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609875c-1b91-4bb3-8846-83ec8cae444d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:12.613777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609875c-1b91-4bb3-8846-83ec8cae444d service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:12.623725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1284/2581] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:12.933098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:12.934627 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:12.955105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4178d-12a5-44be-ad18-326cb1f1db92 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:12.960836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:12.968906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:12.969213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:12.969503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] PATH_INFO: `/v3/projects/c6be7ddcf75e41b2ac131be45afe90c1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:12.971521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6be7ddcf75e41b2ac131be45afe90c1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:12.973520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.036351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6be7ddcf75e41b2ac131be45afe90c1', 'request_id': 'req-e9905928-12fe-4481-b62e-38d83e187e9c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:13.047249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6be7ddcf75e41b2ac131be45afe90c1', 'request_id': 'req-e9905928-12fe-4481-b62e-38d83e187e9c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:13.062694 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6be7ddcf75e41b2ac131be45afe90c1', 'request_id': 'req-e9905928-12fe-4481-b62e-38d83e187e9c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:13.085823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] The token cache is being invalidate because project c6be7ddcf75e41b2ac131be45afe90c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:13.086274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:13.087238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e9905928-12fe-4481-b62e-38d83e187e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:13.092819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1298/2582] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:12 2022] DELETE /identity/v3/projects/c6be7ddcf75e41b2ac131be45afe90c1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:13.094323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.095214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.095512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.140017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:13.141447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:13.235540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1285/2583] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:13.244190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.244482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.244780 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.266611 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:13.267844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:13.275728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1299/2584] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:13.279992 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.289908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc2a82-4aed-4466-b6a2-7bbba5d283d8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.294442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.295174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.295458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.295731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.296459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:13.296766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.298892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.315422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1286/2585] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:40:13.327487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.336661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e517-1638-4c22-b4b2-4d3bb2f8802f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.342186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.342941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.343259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.343539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.344264 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.346890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.374339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1300/2586] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:13 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:13.379351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.389605 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e259018-e896-407d-b0ab-740deae18f21 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.453857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.453857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.453857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.453857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.454678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.454678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.475501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1287/2587] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:13 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:13.479504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.490666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6612c7-75b2-4fb5-843c-efd9da5d0b4b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.495541 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.496201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.496479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.496749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.497471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.499046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.511774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1301/2588] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:40:13.515610 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.521584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:13.528541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe089115-a5f7-4785-9044-e75a21e0fa66 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.535669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c36e6-5677-4964-955c-4f4c085cbd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.536357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c36e6-5677-4964-955c-4f4c085cbd93 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.536635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c36e6-5677-4964-955c-4f4c085cbd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.536908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c36e6-5677-4964-955c-4f4c085cbd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.537652 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c36e6-5677-4964-955c-4f4c085cbd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.539313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c36e6-5677-4964-955c-4f4c085cbd93 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.551860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1288/2589] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:13.556202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.567504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce241-3242-444a-b541-28b694470ab1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.572235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.573152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.573429 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.573699 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] PATH_INFO: `/v3/projects/c04d723b58b749a6ae04bb07771a0df5/users/797d6d8d7cfb47ad8390be3abeef9b0d/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.589141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04d723b58b749a6ae04bb07771a0df5, user_id=797d6d8d7cfb47ad8390be3abeef9b0d, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.591982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.613364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1302/2590] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:13 2022] PUT /identity/v3/projects/c04d723b58b749a6ae04bb07771a0df5/users/797d6d8d7cfb47ad8390be3abeef9b0d/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:13.619032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c4d58-6242-4b11-9345-643afd723df2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.619351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c4d58-6242-4b11-9345-643afd723df2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.619637 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c4d58-6242-4b11-9345-643afd723df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.638846 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9d5c4d58-6242-4b11-9345-643afd723df2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:13.640392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9d5c4d58-6242-4b11-9345-643afd723df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:13.712337 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1289/2591] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:40:13.722610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:13.724410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:13.769641 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:13.788368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6cf99-b04b-4995-9a2b-b444344cb8f5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:13.792311 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:13.793216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:13.793373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:13.793871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:13.807568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:13.809277 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:13.820925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1303/2592] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:14.512555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:14.513113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:14.513430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:14.551345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] MFA Rules not processed for user `797d6d8d7cfb47ad8390be3abeef9b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:14.552874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:14.638154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1290/2593] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:14.639001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:14.650162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48830-a954-4084-ab9f-1d2c789cb98b service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:14.655535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:14.662028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:14.662715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:14.663198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:14.665503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:14.667500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:14.702934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1304/2594] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:14 2022] POST /identity/v3/projects => generated 405 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:40:14.707350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:14.721293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188c493-8d3c-4841-8ddd-c0f36bee5821 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:14.785469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:14.786160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:14.786445 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:14.786754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:14.787502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:14.790035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:14.814238 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1291/2595] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:14 2022] POST /identity/v3/users => generated 309 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:14.818402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:14.830163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1627b-b856-4b6f-9cb9-c66c2eede389 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:14.835566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:14.836280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:14.836571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:14.836851 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:14.837597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:14.839255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:14.852710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1305/2596] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:14.857077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:14.871433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ee103-2bc9-46be-ad99-e2d44370b96f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:14.876338 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:14.877353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:14.877894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:14.878043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] PATH_INFO: `/v3/projects/41b60faad69d4b40aac023785481aea8/users/abeea287e75b4ffba3c78a856f7700db/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:14.888034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:14.897861 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b60faad69d4b40aac023785481aea8, user_id=abeea287e75b4ffba3c78a856f7700db, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:14.902630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:14.931444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1292/2597] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:14 2022] PUT /identity/v3/projects/41b60faad69d4b40aac023785481aea8/users/abeea287e75b4ffba3c78a856f7700db/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:14.933810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:14.947339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737c420-7327-49fa-9a6a-b43bcda54d27 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:15.018418 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383dbd-97fd-4bf2-af0c-682deb6a7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:15.019168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383dbd-97fd-4bf2-af0c-682deb6a7baa admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:15.019460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383dbd-97fd-4bf2-af0c-682deb6a7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:15.019739 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383dbd-97fd-4bf2-af0c-682deb6a7baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:15.020710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383dbd-97fd-4bf2-af0c-682deb6a7baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:15.022271 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383dbd-97fd-4bf2-af0c-682deb6a7baa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:15.038423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1306/2598] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:15.042390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:15.054231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa93b-306a-48ef-8c98-6c3e4405d640 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:15.063093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466787-d58b-4610-9f3a-2ff1b1800fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:15.066219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466787-d58b-4610-9f3a-2ff1b1800fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:15.066510 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466787-d58b-4610-9f3a-2ff1b1800fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:15.066794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466787-d58b-4610-9f3a-2ff1b1800fab admin admin] PATH_INFO: `/v3/projects/41b60faad69d4b40aac023785481aea8/users/abeea287e75b4ffba3c78a856f7700db/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:15.068949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466787-d58b-4610-9f3a-2ff1b1800fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b60faad69d4b40aac023785481aea8, user_id=abeea287e75b4ffba3c78a856f7700db, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:15.071724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466787-d58b-4610-9f3a-2ff1b1800fab admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:15.083091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1293/2599] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:15 2022] PUT /identity/v3/projects/41b60faad69d4b40aac023785481aea8/users/abeea287e75b4ffba3c78a856f7700db/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:16.187133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:16.188305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:16.189287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:16.225332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] MFA Rules not processed for user `abeea287e75b4ffba3c78a856f7700db`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:16.227565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:16.365941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1307/2600] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:17.403469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:17.404017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:17.404017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:17.426094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] MFA Rules not processed for user `d84260bc6c714b7da93086841746a53a`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:17.427516 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:17.462159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:17.505024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:17.507804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1294/2601] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:17.508934 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:17.538495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39a6e7-f167-4b60-a871-ebc80c1f3254 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:17.542706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:17.543476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:17.543968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:17.544303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:17.546154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:17.580825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea83b5-5001-487d-8564-f6536e40b862 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:17.585995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:17.587718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:17.588002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:17.588308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] PATH_INFO: `/v3/users/797d6d8d7cfb47ad8390be3abeef9b0d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:17.589544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] RBAC: Authorizing `identity:delete_user(user_id=797d6d8d7cfb47ad8390be3abeef9b0d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:17.592226 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:17.619599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:17.621044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:17.633609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1308/2602] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:40:17.653110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797d6d8d7cfb47ad8390be3abeef9b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:17.665076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797d6d8d7cfb47ad8390be3abeef9b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:17.667740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797d6d8d7cfb47ad8390be3abeef9b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:17.678862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797d6d8d7cfb47ad8390be3abeef9b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:17.697972 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797d6d8d7cfb47ad8390be3abeef9b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:17.701059 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:17.713476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1295/2603] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:17 2022] DELETE /identity/v3/users/797d6d8d7cfb47ad8390be3abeef9b0d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:18.092778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:18.107945 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89883b-ae6b-4bf3-a529-84952b97b9f6 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:18.185357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:18.186136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:18.186417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:18.186703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] PATH_INFO: `/v3/projects/c04d723b58b749a6ae04bb07771a0df5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:18.187938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04d723b58b749a6ae04bb07771a0df5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:18.190709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:18.262476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04d723b58b749a6ae04bb07771a0df5', 'request_id': 'req-10578c17-7e9e-4b4a-a775-8681b0928b0e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.282059 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04d723b58b749a6ae04bb07771a0df5', 'request_id': 'req-10578c17-7e9e-4b4a-a775-8681b0928b0e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.291297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04d723b58b749a6ae04bb07771a0df5', 'request_id': 'req-10578c17-7e9e-4b4a-a775-8681b0928b0e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.316313 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] The token cache is being invalidate because project c04d723b58b749a6ae04bb07771a0df5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:18.316811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.321969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.323674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1309/2604] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:18 2022] DELETE /identity/v3/projects/c04d723b58b749a6ae04bb07771a0df5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:18.329039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:18.331251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:18.346682 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51ed07-b28f-4163-a7df-cc433855714d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:18.423569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:18.424660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:18.424942 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:18.425219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] PATH_INFO: `/v3/users/abeea287e75b4ffba3c78a856f7700db` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:18.426524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abeea287e75b4ffba3c78a856f7700db)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:18.428541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:18.486821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abeea287e75b4ffba3c78a856f7700db'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.498860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abeea287e75b4ffba3c78a856f7700db'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.502640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abeea287e75b4ffba3c78a856f7700db'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.513625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abeea287e75b4ffba3c78a856f7700db'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.529068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2e419ec-8298-4c5e-89e3-f63426cdd924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abeea287e75b4ffba3c78a856f7700db'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:18.536782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1296/2605] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:18 2022] DELETE /identity/v3/users/abeea287e75b4ffba3c78a856f7700db => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:18.900804 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:18.902408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:18.916996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10578c17-7e9e-4b4a-a775-8681b0928b0e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:18.928180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:19.000150 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.001090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.001379 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.001659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] PATH_INFO: `/v3/projects/41b60faad69d4b40aac023785481aea8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.002871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b60faad69d4b40aac023785481aea8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.004847 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.056232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b60faad69d4b40aac023785481aea8', 'request_id': 'req-4b80020e-08e9-480f-aeb2-2b497f3e66a6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:19.062791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b60faad69d4b40aac023785481aea8', 'request_id': 'req-4b80020e-08e9-480f-aeb2-2b497f3e66a6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:19.069956 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b60faad69d4b40aac023785481aea8', 'request_id': 'req-4b80020e-08e9-480f-aeb2-2b497f3e66a6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:19.086204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] The token cache is being invalidate because project 41b60faad69d4b40aac023785481aea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:19.086485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:19.087324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4b80020e-08e9-480f-aeb2-2b497f3e66a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:19.088928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1310/2606] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:18 2022] DELETE /identity/v3/projects/41b60faad69d4b40aac023785481aea8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:19.094050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.094345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.094632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.116021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:19.117414 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:19.203793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1297/2607] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:40:19.212419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.212924 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.213314 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.238855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:19.240116 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:19.250155 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1311/2608] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:40:19.254560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.266170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8118-37e3-47bb-a44b-f3b63478ae7a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.274049 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.274776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.275196 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.275558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.276467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:19.276857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.278629 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.293607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1298/2609] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:40:19.298218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.308582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a654f-1905-49c5-803f-bd6b2fda2df6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.314048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.314755 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.315009 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.315621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.316021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.318204 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.359616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1312/2610] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:19.361870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.371190 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:19.375367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352646c-ebd8-413d-8283-ca8baf8b80f3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.445852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.446547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.446834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.447163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.447869 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.449820 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.475185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1299/2611] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:19.477603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.489388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaf266-4ffa-4c15-a511-b27324a9ce8e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.494226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.495006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.495395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.495769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.496573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.498296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.511502 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1313/2612] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:40:19.515792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.526940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b1204-266b-439b-b1e8-b040df66e2a8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.531783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d360d-759a-4884-a908-7411a0c221c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.533234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d360d-759a-4884-a908-7411a0c221c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.533528 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d360d-759a-4884-a908-7411a0c221c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.533810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d360d-759a-4884-a908-7411a0c221c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.534826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d360d-759a-4884-a908-7411a0c221c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.536479 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d360d-759a-4884-a908-7411a0c221c3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.547617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1300/2613] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:40:19.552136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.562968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48941bae-b895-4995-b76c-f535cb311183 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.568395 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.569323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.569727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.569990 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] PATH_INFO: `/v3/projects/ef2472c044804e98938c33152013e17b/users/3f21fd827d3a4f0eb26f3ce7a596c2b2/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.586561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2472c044804e98938c33152013e17b, user_id=3f21fd827d3a4f0eb26f3ce7a596c2b2, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.589278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.604549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1314/2614] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:19 2022] PUT /identity/v3/projects/ef2472c044804e98938c33152013e17b/users/3f21fd827d3a4f0eb26f3ce7a596c2b2/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:19.616139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.616574 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.616929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.647431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] MFA Rules not processed for user `3f21fd827d3a4f0eb26f3ce7a596c2b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:19.648647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:19.722741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1301/2615] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:19.727230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.781919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:19.828048 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dd451-4a80-4ac3-b766-22cab292a45c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:19.848171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:19.848959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:19.849248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:19.856071 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:19.856863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:19.860643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:19.905891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1315/2616] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:19.909096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:19.925130 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43ebd4-42e0-49d7-9faf-c01ffe2ab454 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:20.016754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190db89-e913-441e-9059-4614e110240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:20.017860 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190db89-e913-441e-9059-4614e110240d admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.018742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190db89-e913-441e-9059-4614e110240d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.020209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190db89-e913-441e-9059-4614e110240d admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.021421 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190db89-e913-441e-9059-4614e110240d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:20.025629 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190db89-e913-441e-9059-4614e110240d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:20.069973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1302/2617] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:19 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:20.071356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:20.089083 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ce384-5989-42a6-8068-a66d0f5bc75d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:20.096938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:20.098008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.098515 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.099243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.100247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:20.102317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:20.121746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1316/2618] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:20.125867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190db89-e913-441e-9059-4614e110240d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:20.133581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:20.146353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190db89-e913-441e-9059-4614e110240d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:20.154076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:20.155552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.156222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.156758 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] PATH_INFO: `/v3/projects/455ca2b236894311bb596aacff5fb0de/users/ee0d0ec6bb564f489f89744700c0f184/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.180316 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=455ca2b236894311bb596aacff5fb0de, user_id=ee0d0ec6bb564f489f89744700c0f184, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:20.184949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:20.206991 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1303/2619] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:20 2022] PUT /identity/v3/projects/455ca2b236894311bb596aacff5fb0de/users/ee0d0ec6bb564f489f89744700c0f184/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:20.212012 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:20.224974 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04fa2e-63d7-413d-8871-04575d2f6ad9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:20.316538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf023b2a-2773-48ab-9642-5cf0cdf76f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:20.317241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf023b2a-2773-48ab-9642-5cf0cdf76f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.317525 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf023b2a-2773-48ab-9642-5cf0cdf76f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.317959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf023b2a-2773-48ab-9642-5cf0cdf76f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.319539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf023b2a-2773-48ab-9642-5cf0cdf76f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:20.320572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf023b2a-2773-48ab-9642-5cf0cdf76f8c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:20.343126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1317/2620] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:20.350057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:20.390100 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e964d-d205-4c38-a8ce-b814335607ba admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:20.398060 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:20.399373 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.399761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.400273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] PATH_INFO: `/v3/projects/455ca2b236894311bb596aacff5fb0de/users/ee0d0ec6bb564f489f89744700c0f184/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.402801 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455ca2b236894311bb596aacff5fb0de, user_id=ee0d0ec6bb564f489f89744700c0f184, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:20.406440 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:20.430644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.431135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.431420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.449283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1304/2621] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:20 2022] PUT /identity/v3/projects/455ca2b236894311bb596aacff5fb0de/users/ee0d0ec6bb564f489f89744700c0f184/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:20.496388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] MFA Rules not processed for user `ee0d0ec6bb564f489f89744700c0f184`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:20.497635 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:20.635330 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1318/2622] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:20.651004 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:20.653865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:20.730243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10e7b5-e610-4975-ad73-b182698192d8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:20.733944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:20.734969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:20.734969 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:20.735187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:20.825401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:20.826878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:20.837894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1305/2623] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:22.118288 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:22.119819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:22.155807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:22.186829 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b366-53f6-4698-bde8-32aacb2b5070 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:22.191949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:22.192815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.193097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.193371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] PATH_INFO: `/v3/users/a6146c8083d04171863a4b603fd6a46c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.194571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6146c8083d04171863a4b603fd6a46c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:22.196516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:22.242969 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6146c8083d04171863a4b603fd6a46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.252026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6146c8083d04171863a4b603fd6a46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.264605 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6146c8083d04171863a4b603fd6a46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.265275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6146c8083d04171863a4b603fd6a46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.272743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-409ccdb6-e458-4a02-a4c4-af56e69a1430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6146c8083d04171863a4b603fd6a46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.278580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1319/2624] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:22 2022] DELETE /identity/v3/users/a6146c8083d04171863a4b603fd6a46c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:22.492780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:22.508592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f73d3-6ec5-4d5c-aa54-f607ceb77ee6 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:22.605102 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:22.605962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.606240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.606511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] PATH_INFO: `/v3/projects/3b9849b45b2d414797c0cb368b787ddf` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.607713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9849b45b2d414797c0cb368b787ddf)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:22.610357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:22.634163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:22.665819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9849b45b2d414797c0cb368b787ddf', 'request_id': 'req-eb1123f0-0629-4eea-bf3e-f238a6f8d078'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.674576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9849b45b2d414797c0cb368b787ddf', 'request_id': 'req-eb1123f0-0629-4eea-bf3e-f238a6f8d078'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.681264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9849b45b2d414797c0cb368b787ddf', 'request_id': 'req-eb1123f0-0629-4eea-bf3e-f238a6f8d078'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.691876 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] The token cache is being invalidate because project 3b9849b45b2d414797c0cb368b787ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:22.692266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.692832 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb1123f0-0629-4eea-bf3e-f238a6f8d078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:22.694666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1306/2625] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:22 2022] DELETE /identity/v3/projects/3b9849b45b2d414797c0cb368b787ddf => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:22.699342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b043b-3d54-4c93-913b-fde284953110 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.699633 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b043b-3d54-4c93-913b-fde284953110 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.699907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b043b-3d54-4c93-913b-fde284953110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.720094 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-085b043b-3d54-4c93-913b-fde284953110 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:22.721307 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-085b043b-3d54-4c93-913b-fde284953110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:22.787328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1320/2626] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:22.795508 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.795821 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.796124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.816754 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:22.817993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:22.826982 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1307/2627] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:22.831824 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b043b-3d54-4c93-913b-fde284953110 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:22.842029 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b043b-3d54-4c93-913b-fde284953110 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:22.847348 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:22.848015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.848304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.848577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.849302 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:22.849612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:22.851278 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:22.863839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1321/2628] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:40:22.868024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:22.877548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e27b-b0f2-4d7c-855c-c592ae44bc82 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:22.882210 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:22.882857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.883178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.883462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.884211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:22.886084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:22.912467 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1308/2629] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:22 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:22.916808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:22.927773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1ff89-f1a0-4a86-9a31-833aae090442 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:22.965412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:22.986683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:22.987394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:22.987674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:22.987948 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:22.988625 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:22.990538 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:23.010298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1322/2630] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:22 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:23.014788 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:23.026847 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae56c4-00f0-42ab-85eb-b3909f59e545 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:23.031762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:23.033211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:23.033651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:23.033935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:23.034678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:23.036336 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:23.047447 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1309/2631] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:23.052287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:23.062658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7a02-04d7-438d-83c6-706b240b2367 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:23.067775 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212352d-9ac0-4c75-9e84-bdaa7a245d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:23.068499 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212352d-9ac0-4c75-9e84-bdaa7a245d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:23.068899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212352d-9ac0-4c75-9e84-bdaa7a245d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:23.069294 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212352d-9ac0-4c75-9e84-bdaa7a245d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:23.070038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212352d-9ac0-4c75-9e84-bdaa7a245d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:23.071708 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212352d-9ac0-4c75-9e84-bdaa7a245d53 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:23.082927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1323/2632] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:40:23.087416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:23.099088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bdde7-3ac6-41ad-9d24-1832dc8d0202 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:23.104696 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:23.105617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:23.105935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:23.106214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] PATH_INFO: `/v3/projects/68bd5ea42b6f40b89decb32180fae98c/users/4f37aec0e2b64f64a6eb524e39f49b49/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:23.124149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68bd5ea42b6f40b89decb32180fae98c, user_id=4f37aec0e2b64f64a6eb524e39f49b49, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:23.127470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:23.144779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1310/2633] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:23 2022] PUT /identity/v3/projects/68bd5ea42b6f40b89decb32180fae98c/users/4f37aec0e2b64f64a6eb524e39f49b49/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:23.155086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:23.155639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:23.155774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:23.183537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] MFA Rules not processed for user `4f37aec0e2b64f64a6eb524e39f49b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:23.184880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:23.253090 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1324/2634] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:40:23.264386 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:23.266115 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:23.342400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de1919-ddee-4dcb-9c03-c0d3198ccbc7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:23.349140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:23.349810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:23.350098 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:23.350418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:23.360866 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:23.362462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:23.371735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1311/2635] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:25.666399 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:25.667915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:25.728532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f381ea0-dd67-4f89-ac9f-63836b6e0f5d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:25.732105 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:25.732760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:25.733047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:25.733324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:25.735345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:25.801951 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:25.803444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4c16f-75aa-4a5a-ac1b-f247b98dab73 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:25.807587 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:25.815598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1325/2636] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:27.129588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:27.161003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:27.192975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858ead5-6506-43fb-90d1-ceaed4bdb5f5 service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:27.197889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:27.198773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:27.199072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:27.199371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] PATH_INFO: `/v3/users/4f37aec0e2b64f64a6eb524e39f49b49` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:27.200601 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f37aec0e2b64f64a6eb524e39f49b49)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:27.202547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:27.252630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f37aec0e2b64f64a6eb524e39f49b49'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.261770 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f37aec0e2b64f64a6eb524e39f49b49'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.262513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f37aec0e2b64f64a6eb524e39f49b49'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.271072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f37aec0e2b64f64a6eb524e39f49b49'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.286052 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f37aec0e2b64f64a6eb524e39f49b49'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.297507 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1312/2637] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:27 2022] DELETE /identity/v3/users/4f37aec0e2b64f64a6eb524e39f49b49 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:27.303232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:27.303733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:27.303827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:27.324141 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:27.325357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:27.393628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1326/2638] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:27.403153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:27.404650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:27.475683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0f616-da1b-4d45-aa58-b044a51432a9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:27.479619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e9634-dc96-48c5-b187-5744e9d6cbf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:27.480867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e9634-dc96-48c5-b187-5744e9d6cbf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:27.480867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e9634-dc96-48c5-b187-5744e9d6cbf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:27.481121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e9634-dc96-48c5-b187-5744e9d6cbf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:27.491493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e9634-dc96-48c5-b187-5744e9d6cbf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:27.492949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e9634-dc96-48c5-b187-5744e9d6cbf0 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:27.503520 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1313/2639] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:27.778663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:27.780195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:27.793491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f2b9-bfa6-4fae-995d-7ca40c3cb193 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:27.798618 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:27.799534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:27.799817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:27.800152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] PATH_INFO: `/v3/projects/68bd5ea42b6f40b89decb32180fae98c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:27.801469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68bd5ea42b6f40b89decb32180fae98c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:27.803526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:27.856374 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68bd5ea42b6f40b89decb32180fae98c', 'request_id': 'req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.863551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68bd5ea42b6f40b89decb32180fae98c', 'request_id': 'req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.871050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68bd5ea42b6f40b89decb32180fae98c', 'request_id': 'req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.884746 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] The token cache is being invalidate because project 68bd5ea42b6f40b89decb32180fae98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:27.884964 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.885563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-682c0beb-ca2b-49e2-8b6a-4d8c37309a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:27.887817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1327/2640] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:27 2022] DELETE /identity/v3/projects/68bd5ea42b6f40b89decb32180fae98c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:27.895073 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:27.895358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:27.895643 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:27.913987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:27.915333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:27.965462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:27.980571 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1314/2641] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:27.989520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:27.989805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:27.990088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.009923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:28.011193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:28.020590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1328/2642] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:28.024908 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.035521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f405a-f52d-433b-a191-44b92717aeec None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.041258 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.041943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.042400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.043404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.043557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:28.043789 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.045505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.061347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1315/2643] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:40:28.062807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.073569 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da69cc-7a0b-44eb-9a19-dce3aa136bc1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.078249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.079545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.079833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.080106 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.080803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.082723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.109166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1329/2644] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:28 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:40:28.113661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.123689 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873afea-ae8b-4f2a-b282-283ad8ec0c3c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.183269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.184010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.184119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.184394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.185086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.187021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.216784 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1316/2645] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:28 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:28.218138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.229035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc6a7-6bac-43df-8ee1-35b6a51f0f9f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.233701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.234373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.234662 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.235498 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.236407 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.238021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.249265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1330/2646] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:28.254126 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.264794 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26060d79-4688-4e01-8ce9-96e68f032ba7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.269975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc048c5-496d-4a69-aae3-3daf1f177b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.270639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc048c5-496d-4a69-aae3-3daf1f177b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.270924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc048c5-496d-4a69-aae3-3daf1f177b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.271235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc048c5-496d-4a69-aae3-3daf1f177b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.271966 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc048c5-496d-4a69-aae3-3daf1f177b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.273559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc048c5-496d-4a69-aae3-3daf1f177b93 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.284129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1317/2647] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:28.288552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.299623 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b00b75-d775-4537-9dca-3a97507ce14d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.304460 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.305355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.305639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.305915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] PATH_INFO: `/v3/projects/17f2c14f6f814f89b35aad3befd1b491/users/2938b6c9a17b4dee8335578222013513/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.321953 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f2c14f6f814f89b35aad3befd1b491, user_id=2938b6c9a17b4dee8335578222013513, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.324732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.345534 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1331/2648] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:28 2022] PUT /identity/v3/projects/17f2c14f6f814f89b35aad3befd1b491/users/2938b6c9a17b4dee8335578222013513/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:28.357439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.357819 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.358165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.388751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] MFA Rules not processed for user `2938b6c9a17b4dee8335578222013513`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:28.390231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:28.455041 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1318/2649] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:40:28.466086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:28.467765 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:28.488257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:28.527160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5baaf-cf25-4780-ab85-079fec1d8609 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:28.530697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0521e-8093-4619-9fe3-e19a7bfaafa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:28.531370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0521e-8093-4619-9fe3-e19a7bfaafa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:28.531660 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0521e-8093-4619-9fe3-e19a7bfaafa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:28.531976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0521e-8093-4619-9fe3-e19a7bfaafa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:28.540909 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0521e-8093-4619-9fe3-e19a7bfaafa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:28.542370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0521e-8093-4619-9fe3-e19a7bfaafa8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:28.552996 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1332/2650] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:31.097433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:31.146804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:31.163205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515134e8-d3fb-4464-acb4-bd557c3339d3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:31.167858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:31.168742 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.169025 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.169300 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] PATH_INFO: `/v3/users/2938b6c9a17b4dee8335578222013513` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.170529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2938b6c9a17b4dee8335578222013513)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:31.172522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:31.221532 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2938b6c9a17b4dee8335578222013513'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.229035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2938b6c9a17b4dee8335578222013513'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.229665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2938b6c9a17b4dee8335578222013513'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.237214 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2938b6c9a17b4dee8335578222013513'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.249077 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2938b6c9a17b4dee8335578222013513'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.256159 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1319/2651] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:31 2022] DELETE /identity/v3/users/2938b6c9a17b4dee8335578222013513 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:31.260732 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.261248 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.261364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.279016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:31.280254 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:31.338710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1333/2652] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:40:31.348399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:31.349811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:31.407984 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa093c0-ed96-4456-a748-fedb03a8b24f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:31.411686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850aac-2def-49d2-8b58-59ddf4731736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:31.412358 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850aac-2def-49d2-8b58-59ddf4731736 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.412623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850aac-2def-49d2-8b58-59ddf4731736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.412902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850aac-2def-49d2-8b58-59ddf4731736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.422011 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d850aac-2def-49d2-8b58-59ddf4731736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:31.424009 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d850aac-2def-49d2-8b58-59ddf4731736 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:31.433598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1320/2653] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:31.723642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:31.725181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:31.737529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c56cc8-780c-4cde-a90e-6178899ce2b7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:31.742249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:31.743355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.743642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.743921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] PATH_INFO: `/v3/projects/17f2c14f6f814f89b35aad3befd1b491` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.745170 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f2c14f6f814f89b35aad3befd1b491)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:31.747163 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:31.800229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f2c14f6f814f89b35aad3befd1b491', 'request_id': 'req-c85325a6-f0d4-468a-837c-8bdea055d3b7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.806304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f2c14f6f814f89b35aad3befd1b491', 'request_id': 'req-c85325a6-f0d4-468a-837c-8bdea055d3b7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.812027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f2c14f6f814f89b35aad3befd1b491', 'request_id': 'req-c85325a6-f0d4-468a-837c-8bdea055d3b7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.824614 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] The token cache is being invalidate because project 17f2c14f6f814f89b35aad3befd1b491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:31.824902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.825554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c85325a6-f0d4-468a-837c-8bdea055d3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:31.827136 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1334/2654] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:31 2022] DELETE /identity/v3/projects/17f2c14f6f814f89b35aad3befd1b491 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:31.832631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.832919 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.833199 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.850579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:31.851814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:31.913222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1321/2655] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:31.921168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.921442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.921716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.941312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:31.942503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:31.950394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1335/2656] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:31.954631 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:31.964408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcce00-e186-45a1-a071-dd1a45dfa06e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:31.969043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:31.969718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:31.970029 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:31.970306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:31.971047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:31.971383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:31.973076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:31.980008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:31.987728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1322/2657] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:40:31.991921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:32.001823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd016f-99f5-4660-9a4a-47e78342255b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:32.006685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:32.007377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.007664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.007942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.008646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:32.010654 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:32.037152 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1336/2658] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:31 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:32.041120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:32.051751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b0a72-ecea-4ff3-a300-7ca783ee2698 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:32.109371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:32.110165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.110981 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.111306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.112099 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:32.114042 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:32.137461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1323/2659] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:32 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:40:32.139695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:32.150050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e226-5fe7-458b-b027-c3e5f085d3fc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:32.154784 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:32.155488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.155774 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.156051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.156782 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:32.158595 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:32.169915 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1337/2660] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:32.174939 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:32.185303 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439ffb6-18dc-42a2-8eb6-7edf800706ab admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:32.190834 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003f85-2152-4f37-8d56-de1f1bdb5f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:32.191546 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003f85-2152-4f37-8d56-de1f1bdb5f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.191844 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003f85-2152-4f37-8d56-de1f1bdb5f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.192123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003f85-2152-4f37-8d56-de1f1bdb5f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.192857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60003f85-2152-4f37-8d56-de1f1bdb5f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:32.194468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60003f85-2152-4f37-8d56-de1f1bdb5f35 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:32.204945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1324/2661] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:32.209272 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:32.216485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:32.221171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912bdda-3cac-458b-b974-c594d25754bb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:32.229068 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:32.230783 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.231263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.231710 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] PATH_INFO: `/v3/projects/3254e6fd0ec64aff9f4625e4ea9cfcb1/users/4a33eede53e4462aab5c24ad0eaa9abb/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.247821 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3254e6fd0ec64aff9f4625e4ea9cfcb1, user_id=4a33eede53e4462aab5c24ad0eaa9abb, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:32.250510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:32.264543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1338/2662] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:32 2022] PUT /identity/v3/projects/3254e6fd0ec64aff9f4625e4ea9cfcb1/users/4a33eede53e4462aab5c24ad0eaa9abb/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:32.275980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1691-d246-47d8-afb5-afc4a92f28b5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.276250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1691-d246-47d8-afb5-afc4a92f28b5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.276554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1691-d246-47d8-afb5-afc4a92f28b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.303504 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-46ff1691-d246-47d8-afb5-afc4a92f28b5 None None] MFA Rules not processed for user `4a33eede53e4462aab5c24ad0eaa9abb`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:32.304641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-46ff1691-d246-47d8-afb5-afc4a92f28b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:32.376324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1325/2663] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:32.387528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:32.389494 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:32.446701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1a959-e256-4f83-8f00-54e3350cd5bc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:32.450636 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3df4115fcce342f2ae095ec1a2af4cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:32.451345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:32.451631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:32.451916 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:32.460786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:32.462263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:32.471665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1339/2664] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:40:42.373298 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:42.467419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:44.321329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:44.321838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:44.322021 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:44.350914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:44.352793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:44.443161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1326/2665] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:40:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:44.500354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:44.502007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:44.615415 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948c467-ed3d-4daa-9cf4-54115da2fb02 service glance] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:44.621769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330b961-ed69-4393-a747-7b00446be50c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:44.622442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330b961-ed69-4393-a747-7b00446be50c service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:44.622722 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330b961-ed69-4393-a747-7b00446be50c service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:44.623000 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330b961-ed69-4393-a747-7b00446be50c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:44.632113 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330b961-ed69-4393-a747-7b00446be50c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:44.634037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330b961-ed69-4393-a747-7b00446be50c service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:44.642867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1340/2666] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:40:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:40:53.327660 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:53.331259 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:53.500545 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a7b29-0952-4518-b4ba-e980962c57cc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:53.506211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:53.508929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:53.508929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:53.508929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] PATH_INFO: `/v3/users/3f21fd827d3a4f0eb26f3ce7a596c2b2` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:53.510443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f21fd827d3a4f0eb26f3ce7a596c2b2)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:53.513916 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:53.580717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:53.584296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f21fd827d3a4f0eb26f3ce7a596c2b2'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:53.598694 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f21fd827d3a4f0eb26f3ce7a596c2b2'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:53.599603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f21fd827d3a4f0eb26f3ce7a596c2b2'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:53.609705 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f21fd827d3a4f0eb26f3ce7a596c2b2'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:53.632664 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f21fd827d3a4f0eb26f3ce7a596c2b2'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:53.640585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1327/2667] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:53 2022] DELETE /identity/v3/users/3f21fd827d3a4f0eb26f3ce7a596c2b2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:53.645805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:53.646457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:53.646731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:53.693199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:53.704086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:53.791340 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1341/2668] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:53.807967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:53.810144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:53.906684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb1b8c-5a7b-413a-a113-bb5acc112c42 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:53.914175 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:53.915244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:53.917001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:53.917001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:53.930968 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:53.936260 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:53.969706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1328/2669] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:54.358081 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:54.361573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:54.376037 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f65157-72c4-4b69-950d-ec19b1680515 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:54.383475 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:54.384709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:54.385074 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:54.385414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] PATH_INFO: `/v3/projects/ef2472c044804e98938c33152013e17b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:54.387208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2472c044804e98938c33152013e17b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:54.389866 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:54.457122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:54.458607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2472c044804e98938c33152013e17b', 'request_id': 'req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.469190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2472c044804e98938c33152013e17b', 'request_id': 'req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.476930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2472c044804e98938c33152013e17b', 'request_id': 'req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.497987 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] The token cache is being invalidate because project ef2472c044804e98938c33152013e17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:54.498263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.502794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.503983 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1342/2670] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:54 2022] DELETE /identity/v3/projects/ef2472c044804e98938c33152013e17b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:54.505271 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:54.520091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329a890-d3db-4d92-84f4-34d3401ec1f9 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:54.610713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:54.612000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:54.612286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:54.612562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] PATH_INFO: `/v3/users/ee0d0ec6bb564f489f89744700c0f184` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:54.613795 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0d0ec6bb564f489f89744700c0f184)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:54.615836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:54.683328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0d0ec6bb564f489f89744700c0f184'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.702348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0d0ec6bb564f489f89744700c0f184'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.703080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0d0ec6bb564f489f89744700c0f184'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.717778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0d0ec6bb564f489f89744700c0f184'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.729635 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:54.749346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7353725-f324-4bc5-a532-6017278c8b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0d0ec6bb564f489f89744700c0f184'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:54.757296 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1329/2671] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:40:54 2022] DELETE /identity/v3/users/ee0d0ec6bb564f489f89744700c0f184 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:55.113873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.115491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:55.132863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f195e-266b-4c98-9b9d-e4ccadb24fa7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.215589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.216594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.216931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.217280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] PATH_INFO: `/v3/projects/455ca2b236894311bb596aacff5fb0de` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.218583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=455ca2b236894311bb596aacff5fb0de)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.220674 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.282560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455ca2b236894311bb596aacff5fb0de', 'request_id': 'req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:55.294026 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455ca2b236894311bb596aacff5fb0de', 'request_id': 'req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:55.302481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455ca2b236894311bb596aacff5fb0de', 'request_id': 'req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:55.316949 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] The token cache is being invalidate because project 455ca2b236894311bb596aacff5fb0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:40:55.317334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:55.318056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfa7ab17-35ea-40c5-b0ed-815b3676a8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:40:55.323815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1343/2672] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:40:55 2022] DELETE /identity/v3/projects/455ca2b236894311bb596aacff5fb0de => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:55.325733 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.326062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.326350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.384874 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:55.386137 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:55.463332 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1330/2673] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:55.469457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.469882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.470237 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.504524 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:55.506319 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:55.517051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1344/2674] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:55.521665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.536935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11bc9e-99fe-40e7-b80b-94009788546e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.542777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.543487 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.543773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.544050 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.544785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:40:55.545093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.546791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.570609 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1331/2675] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:40:55.572419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.600069 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05f390-9d45-4037-88b7-f2be8ddd0e9e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.611354 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.612342 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.612631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.612791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.614047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.619033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.679603 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1345/2676] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:55 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:40:55.685012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.698264 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0ada7-ded3-45b1-9d77-ee0f64e7bbc1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.780651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.781576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.782281 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.783431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.783645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.785906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.819588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1332/2677] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:55 2022] POST /identity/v3/users => generated 301 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:55.824661 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.837203 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af975dcb-0ce6-4d9b-87d6-ab83ddd4d2f4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.842991 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.843902 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.844382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.844509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.845334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.847100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.861428 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1346/2678] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:55.867008 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.886409 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2055b9-83b4-4707-b361-413652820488 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.892367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e52ffa-fc32-40de-86f5-2a913b92daad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.893153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52ffa-fc32-40de-86f5-2a913b92daad admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.893535 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52ffa-fc32-40de-86f5-2a913b92daad admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.893882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52ffa-fc32-40de-86f5-2a913b92daad admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.894717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e52ffa-fc32-40de-86f5-2a913b92daad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.896867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e52ffa-fc32-40de-86f5-2a913b92daad admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.912356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1333/2679] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:55.913624 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:55.925669 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfcbf4-bb86-4709-9c2d-e50463703d7f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:55.931867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:55.932799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.933241 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.933526 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] PATH_INFO: `/v3/projects/e2f86357fd194ff9b26f3d8bd001bf24/users/508e37289f18430c926b4c31efd44e6c/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:55.950474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f86357fd194ff9b26f3d8bd001bf24, user_id=508e37289f18430c926b4c31efd44e6c, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:55.953396 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:55.993051 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1347/2680] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:55 2022] PUT /identity/v3/projects/e2f86357fd194ff9b26f3d8bd001bf24/users/508e37289f18430c926b4c31efd44e6c/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:55.995940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:55.996292 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:55.996581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.027657 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] MFA Rules not processed for user `508e37289f18430c926b4c31efd44e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:56.028903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:56.042760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:56.129747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1334/2681] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:56.134092 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.142413 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:56.214787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9621bed-72a9-4cbc-96ed-425db9a01da7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.219695 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.220452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.220808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.221156 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.222112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.224174 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.264167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1348/2682] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:40:56 2022] POST /identity/v3/projects => generated 399 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:40:56.266184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.278475 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756fdcc-9a06-44aa-b3f9-c49b600fb14e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.353852 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.355035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.355379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.355655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.356551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.359013 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.382311 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1335/2683] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:40:56 2022] POST /identity/v3/users => generated 306 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:40:56.386703 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.398528 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afbdc9-dd26-4949-b9ce-48bb6daba76a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.405180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.406763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.407049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.407345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.408086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.409728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.422334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1349/2684] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:40:56.427244 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.441891 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76524fed-41b4-491f-bcbd-186a50590a86 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.447764 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.448690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.448953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.449229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] PATH_INFO: `/v3/projects/529b5e60847c4804b0f80bb6eecee9f2/users/871e575409ac4f61b3416fa226fd80ef/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.467519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=529b5e60847c4804b0f80bb6eecee9f2, user_id=871e575409ac4f61b3416fa226fd80ef, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.470321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.489407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1336/2685] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:56 2022] PUT /identity/v3/projects/529b5e60847c4804b0f80bb6eecee9f2/users/871e575409ac4f61b3416fa226fd80ef/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:40:56.494939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.507120 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd80ff-b620-46ce-aff4-108afe1bd300 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.575876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0403a-921d-495c-9d8a-7945c0c5f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.576736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0403a-921d-495c-9d8a-7945c0c5f2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.577086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0403a-921d-495c-9d8a-7945c0c5f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.577432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0403a-921d-495c-9d8a-7945c0c5f2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.578226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0403a-921d-495c-9d8a-7945c0c5f2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.579979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0403a-921d-495c-9d8a-7945c0c5f2ca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.595257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1350/2686] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:40:56.596416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.607587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f1afb-b0ac-4766-98cb-31214b631673 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.612547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.613448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.613728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.614000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] PATH_INFO: `/v3/projects/529b5e60847c4804b0f80bb6eecee9f2/users/871e575409ac4f61b3416fa226fd80ef/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.616256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=529b5e60847c4804b0f80bb6eecee9f2, user_id=871e575409ac4f61b3416fa226fd80ef, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.619045 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.631364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1337/2687] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:40:56 2022] PUT /identity/v3/projects/529b5e60847c4804b0f80bb6eecee9f2/users/871e575409ac4f61b3416fa226fd80ef/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:40:56.649527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.650745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.651421 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.692817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] MFA Rules not processed for user `871e575409ac4f61b3416fa226fd80ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:40:56.694211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:56.786802 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:56.798265 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1351/2688] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:40:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:40:56.808949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:56.811132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:56.925241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d9206-a0d6-4da0-977b-2f3ab74aef3e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:56.931735 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:56.932900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:56.933408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:56.934351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:56.952737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:56.955805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:56.972928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1338/2689] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:57.080100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:57.092195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30bbf-273f-4564-b555-001d277f0ada None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:57.096036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ef16d-183d-48f5-a48c-2dcd9ca4c547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:57.096692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ef16d-183d-48f5-a48c-2dcd9ca4c547 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:57.096975 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ef16d-183d-48f5-a48c-2dcd9ca4c547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:57.097270 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ef16d-183d-48f5-a48c-2dcd9ca4c547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:57.177731 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ef16d-183d-48f5-a48c-2dcd9ca4c547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:57.179550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ef16d-183d-48f5-a48c-2dcd9ca4c547 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:57.194942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1352/2690] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:40:57.268443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:57.270812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:57.291686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:57.420575 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c177476-fd63-4b02-964b-365ecd192a6e service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:57.426531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:57.427148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:57.427509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:57.427728 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:57.429946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:57.445219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:57.447597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:57.462283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1339/2691] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:00.149817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:00.150433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:00.150433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:00.179124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:00.180341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:00.239877 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1353/2692] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jul 21 22:41:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:00.253625 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:00.255315 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:00.332606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e07945-7aa8-4d50-85a2-446d19fcb87e service placement] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:00.340474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77633201-b4ca-4d1c-b261-f0f05fe5a342 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:00.341141 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633201-b4ca-4d1c-b261-f0f05fe5a342 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:00.341463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633201-b4ca-4d1c-b261-f0f05fe5a342 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:00.341737 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633201-b4ca-4d1c-b261-f0f05fe5a342 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:00.361509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633201-b4ca-4d1c-b261-f0f05fe5a342 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:00.363681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633201-b4ca-4d1c-b261-f0f05fe5a342 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:00.373185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1340/2693] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:41:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:41:01.558015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:01.619852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516f875-e2fa-4d86-8b6d-c638d4081283 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:01.624766 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:01.625643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:01.625927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:01.626202 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] PATH_INFO: `/v3/users/508e37289f18430c926b4c31efd44e6c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:01.627441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508e37289f18430c926b4c31efd44e6c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:01.629377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:01.681977 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:01.691855 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508e37289f18430c926b4c31efd44e6c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:01.700913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508e37289f18430c926b4c31efd44e6c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:01.715172 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508e37289f18430c926b4c31efd44e6c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:01.716178 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508e37289f18430c926b4c31efd44e6c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:01.724970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508e37289f18430c926b4c31efd44e6c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:01.733320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1354/2694] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:01 2022] DELETE /identity/v3/users/508e37289f18430c926b4c31efd44e6c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:01.737922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:01.738404 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:01.738503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:01.756088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:01.757314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:01.834597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1341/2695] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:01.845481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:01.847003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:01.923412 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766cb72-c779-4d02-a817-7fa0647f54a1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:01.940239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:01.940239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:01.940239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:01.940239 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:01.951759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:01.953485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:01.964128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1355/2696] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:02.272480 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:02.273995 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:02.286075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9679f72-5dff-42ec-acdd-becb046d5d98 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:02.291164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:02.292592 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:02.292883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:02.293144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] PATH_INFO: `/v3/projects/e2f86357fd194ff9b26f3d8bd001bf24` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:02.294371 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f86357fd194ff9b26f3d8bd001bf24)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:02.296435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:02.299335 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:02.371243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f86357fd194ff9b26f3d8bd001bf24', 'request_id': 'req-96c537a9-2242-4da8-b0e4-2bbe42c09297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.386661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f86357fd194ff9b26f3d8bd001bf24', 'request_id': 'req-96c537a9-2242-4da8-b0e4-2bbe42c09297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.395339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f86357fd194ff9b26f3d8bd001bf24', 'request_id': 'req-96c537a9-2242-4da8-b0e4-2bbe42c09297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.409511 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] The token cache is being invalidate because project e2f86357fd194ff9b26f3d8bd001bf24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:02.410881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:02.412458 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:02.413168 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.413881 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.416048 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1342/2697] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:02 2022] DELETE /identity/v3/projects/e2f86357fd194ff9b26f3d8bd001bf24 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:02.420681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:02.422665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:02.432707 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007e3b6-1703-4e67-b137-81a9493feeec service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:02.493471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c537a9-2242-4da8-b0e4-2bbe42c09297 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:02.494500 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:02.496180 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:02.497066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:02.497349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] PATH_INFO: `/v3/users/d84260bc6c714b7da93086841746a53a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:02.498549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84260bc6c714b7da93086841746a53a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:02.498752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:02.499669 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:02.499953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:02.500225 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] PATH_INFO: `/v3/users/871e575409ac4f61b3416fa226fd80ef` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:02.500548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:02.501587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871e575409ac4f61b3416fa226fd80ef)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:02.503548 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:02.550441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84260bc6c714b7da93086841746a53a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.550744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871e575409ac4f61b3416fa226fd80ef'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.559844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84260bc6c714b7da93086841746a53a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.562094 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871e575409ac4f61b3416fa226fd80ef'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.562575 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:02.562899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871e575409ac4f61b3416fa226fd80ef'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.570886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871e575409ac4f61b3416fa226fd80ef'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.575592 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84260bc6c714b7da93086841746a53a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.576529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84260bc6c714b7da93086841746a53a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.583463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84260bc6c714b7da93086841746a53a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.583755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871e575409ac4f61b3416fa226fd80ef'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:02.590138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1356/2698] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:02 2022] DELETE /identity/v3/users/d84260bc6c714b7da93086841746a53a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:02.590771 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1343/2699] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:02 2022] DELETE /identity/v3/users/871e575409ac4f61b3416fa226fd80ef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:02.795298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:02.799910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:02.876067 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b81c1-5ee2-40c2-b884-e415ff4072b9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:02.895704 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:02.897816 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:02.912471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:02.960289 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:02.961159 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:02.961442 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:02.961715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] PATH_INFO: `/v3/projects/1d10b882ad80447aa37649632a0d5692` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:02.963511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d10b882ad80447aa37649632a0d5692)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:02.965590 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:02.982581 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7955a3-4efb-4551-a8d2-2a3686ff7d00 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:02.987450 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:02.988365 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:02.988640 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:02.988932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] PATH_INFO: `/v3/projects/529b5e60847c4804b0f80bb6eecee9f2` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:02.990227 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=529b5e60847c4804b0f80bb6eecee9f2)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:02.992218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.020357 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d10b882ad80447aa37649632a0d5692', 'request_id': 'req-9dcc7d06-3571-40af-92f4-dd12fb542649'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.031359 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d10b882ad80447aa37649632a0d5692', 'request_id': 'req-9dcc7d06-3571-40af-92f4-dd12fb542649'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.038533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d10b882ad80447aa37649632a0d5692', 'request_id': 'req-9dcc7d06-3571-40af-92f4-dd12fb542649'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.054656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529b5e60847c4804b0f80bb6eecee9f2', 'request_id': 'req-c1b32926-6229-419f-a6b8-8eae9fa64a2f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.055299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] The token cache is being invalidate because project 1d10b882ad80447aa37649632a0d5692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:03.055594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.056323 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.061520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1357/2700] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:02 2022] DELETE /identity/v3/projects/1d10b882ad80447aa37649632a0d5692 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:03.061520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.061520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.065666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529b5e60847c4804b0f80bb6eecee9f2', 'request_id': 'req-c1b32926-6229-419f-a6b8-8eae9fa64a2f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.072513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529b5e60847c4804b0f80bb6eecee9f2', 'request_id': 'req-c1b32926-6229-419f-a6b8-8eae9fa64a2f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.074794 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:03.085277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] The token cache is being invalidate because project 529b5e60847c4804b0f80bb6eecee9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:03.085569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.086293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c1b32926-6229-419f-a6b8-8eae9fa64a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.088220 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1344/2701] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:02 2022] DELETE /identity/v3/projects/529b5e60847c4804b0f80bb6eecee9f2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:03.089756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb0d12-8395-4c12-b798-ff4e9ecaf550 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.090039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb0d12-8395-4c12-b798-ff4e9ecaf550 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.090314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb0d12-8395-4c12-b798-ff4e9ecaf550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.109688 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-93bb0d12-8395-4c12-b798-ff4e9ecaf550 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.110961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-93bb0d12-8395-4c12-b798-ff4e9ecaf550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.133316 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc7d06-3571-40af-92f4-dd12fb542649 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.173134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1345/2702] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:03.174708 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eafcae-c16c-4c79-ac54-e1296186e3a3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.175269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eafcae-c16c-4c79-ac54-e1296186e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.175416 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eafcae-c16c-4c79-ac54-e1296186e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.190098 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.190899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.191211 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.191488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] PATH_INFO: `/v3/users/4a33eede53e4462aab5c24ad0eaa9abb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.192716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a33eede53e4462aab5c24ad0eaa9abb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.194673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.195618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-31eafcae-c16c-4c79-ac54-e1296186e3a3 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.196867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-31eafcae-c16c-4c79-ac54-e1296186e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.206489 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1346/2703] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:03.207848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3667be6-68ae-4a33-a2c2-08a2f0061300 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.208140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3667be6-68ae-4a33-a2c2-08a2f0061300 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.208415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3667be6-68ae-4a33-a2c2-08a2f0061300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.228286 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c3667be6-68ae-4a33-a2c2-08a2f0061300 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.230237 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c3667be6-68ae-4a33-a2c2-08a2f0061300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.238683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1347/2704] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:03.240053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.240340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.240617 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.254124 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a33eede53e4462aab5c24ad0eaa9abb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.259118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.260343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.261814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a33eede53e4462aab5c24ad0eaa9abb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.269496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1348/2705] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:03.270096 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.275034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a33eede53e4462aab5c24ad0eaa9abb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.275754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a33eede53e4462aab5c24ad0eaa9abb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.280484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7721f9-0959-4b7e-9644-134db026be9f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.283607 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a33eede53e4462aab5c24ad0eaa9abb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:03.285108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.285756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.286039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.286317 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.287039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:41:03.287379 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.288980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e521a-05df-4fce-9a21-7ef2407da4ca admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.289919 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1358/2706] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:03 2022] DELETE /identity/v3/users/4a33eede53e4462aab5c24ad0eaa9abb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:03.292577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.294077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.301205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1349/2707] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:41:03.302544 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.302830 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.303189 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.321377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.322590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.327579 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:03.343873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:03.369231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912a28f-a32f-45e9-a1c1-7559d047f0de admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.374425 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.375599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.375599 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.375831 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.376393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:41:03.376769 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.378383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.391487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1359/2708] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:41:03.392297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.394552 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.396976 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1350/2709] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:03.397792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.409887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902d09f-73d8-401d-b7ff-57bcff84f1d7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.414535 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a6c06-1cf0-429b-83c6-8bec0aaa9d1c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.417269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.418329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.418809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.419418 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.420572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.421773 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.422792 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.423297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.423757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.424075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.424914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.428356 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.457986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1351/2710] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:03.459208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.460133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1360/2711] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:03.462341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.462989 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.524764 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220e6ee-bb44-4764-85cf-d3e2d0bcde86 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.530073 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.530836 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.531088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.531332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.531998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.533931 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.546423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb46475-6134-44fc-86d8-09833f981ca0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.554544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1361/2712] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:03.555284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.565850 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26a35e-290a-4248-a609-2063e3f65012 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.570585 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.571284 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.571576 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.571857 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.572544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.575199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.595828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1362/2713] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:03.596655 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.597001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.597621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.597914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.598192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.606466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a414-9d2d-44f6-a5e6-8eb3283d2cd3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.608057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.609509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.611246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.611905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.612191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.612473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.613199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.614870 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.620773 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1352/2714] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:03.622462 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.626880 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1363/2715] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:03.631310 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.635356 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6bdc-f2bc-4383-8724-7951b088bda3 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.643012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.643232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f30985-66c2-4910-bf45-c5c9a01f647a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.643693 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.644351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.644460 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.645145 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.646962 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.648874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.649548 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.649834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.650115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.650883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.652604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.666656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1364/2716] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:41:03.666870 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1353/2717] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:03.671349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.675279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.685648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16d524-1d21-4cca-a546-70164795f33f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.688407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d17c1-74d5-49c2-b2c5-1619366e423d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.690903 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.691657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.691911 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.692230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.692978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.695027 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.702218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb6c3-f0da-435c-90d7-a4657f816cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.703661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb6c3-f0da-435c-90d7-a4657f816cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.704380 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb6c3-f0da-435c-90d7-a4657f816cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.704858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb6c3-f0da-435c-90d7-a4657f816cd3 admin admin] PATH_INFO: `/v3/projects/6efd5cce97594e9ebbd970aac9557314/users/620fb117ea594c6d893411302e00ab78/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.716439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1365/2718] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:41:03.718467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.724475 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:03.728945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fb6c3-f0da-435c-90d7-a4657f816cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6efd5cce97594e9ebbd970aac9557314, user_id=620fb117ea594c6d893411302e00ab78, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.732618 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fb6c3-f0da-435c-90d7-a4657f816cd3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.734947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc19c-e3d8-4522-bff5-a2cd0c5d6bb4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.740626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fb088-0c82-489a-8e45-5dee4b342a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.741536 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fb088-0c82-489a-8e45-5dee4b342a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.741861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fb088-0c82-489a-8e45-5dee4b342a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.742292 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fb088-0c82-489a-8e45-5dee4b342a5b admin admin] PATH_INFO: `/v3/projects/4d1af09e3e2b40be92caf9c0ef51b099/users/00304ab3aada47ccb0756d40b5484ea3/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.749093 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1354/2719] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:03 2022] PUT /identity/v3/projects/6efd5cce97594e9ebbd970aac9557314/users/620fb117ea594c6d893411302e00ab78/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:03.759381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.759779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.759947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fb088-0c82-489a-8e45-5dee4b342a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1af09e3e2b40be92caf9c0ef51b099, user_id=00304ab3aada47ccb0756d40b5484ea3, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.760080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.763823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fb088-0c82-489a-8e45-5dee4b342a5b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.784135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1366/2720] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:03 2022] PUT /identity/v3/projects/4d1af09e3e2b40be92caf9c0ef51b099/users/00304ab3aada47ccb0756d40b5484ea3/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:03.796138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.796452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.796811 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.804390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] MFA Rules not processed for user `620fb117ea594c6d893411302e00ab78`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.805867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.826572 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] MFA Rules not processed for user `00304ab3aada47ccb0756d40b5484ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:03.829353 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.844882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:03.884329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1355/2721] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:03.890076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.910578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1367/2722] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:03.923371 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:03.925666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:03.969083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2808c-01b0-4dd5-be39-0fe140fee641 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:03.974464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:03.975731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:03.976058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:03.976350 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:03.977153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:03.979012 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:03.997907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd427825-5c3d-49b5-911f-c3ea3c85b1bb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.002243 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.002908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.003234 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.003521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.009293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1356/2723] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:03 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:04.009293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.012593 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.016280 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.018191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.028426 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1368/2724] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:41:04.029298 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.039848 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffb9dd-f6b5-4f73-bb73-cac76a24cf19 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.106496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2be8-e6b8-4308-995b-7bc24cbe5931 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.112634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.113645 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.113938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.114349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] PATH_INFO: `/v3/projects/3254e6fd0ec64aff9f4625e4ea9cfcb1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.115854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3254e6fd0ec64aff9f4625e4ea9cfcb1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.117865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.126612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.127601 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.127799 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.128417 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.128815 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:04.129199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.131562 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.157231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1369/2725] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:04.161511 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.172651 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3200ec3-99f6-40ca-a21e-91336faf6b1a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.180468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3254e6fd0ec64aff9f4625e4ea9cfcb1', 'request_id': 'req-62d70b26-57f1-4a90-9d55-7a04ad5a2843'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:04.190585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3254e6fd0ec64aff9f4625e4ea9cfcb1', 'request_id': 'req-62d70b26-57f1-4a90-9d55-7a04ad5a2843'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:04.201328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3254e6fd0ec64aff9f4625e4ea9cfcb1', 'request_id': 'req-62d70b26-57f1-4a90-9d55-7a04ad5a2843'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:04.221105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] The token cache is being invalidate because project 3254e6fd0ec64aff9f4625e4ea9cfcb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:04.221453 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:04.222466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-62d70b26-57f1-4a90-9d55-7a04ad5a2843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:04.225108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1357/2726] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:04 2022] DELETE /identity/v3/projects/3254e6fd0ec64aff9f4625e4ea9cfcb1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:04.233256 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3da84c-ea0d-4236-abbe-d9d1d75fa26c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.233651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3da84c-ea0d-4236-abbe-d9d1d75fa26c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.233945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3da84c-ea0d-4236-abbe-d9d1d75fa26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.243801 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:04.256205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1a3da84c-ea0d-4236-abbe-d9d1d75fa26c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:04.258148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1a3da84c-ea0d-4236-abbe-d9d1d75fa26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.264972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.266021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.266539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.267003 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.268195 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.270181 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1358/2727] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:04.270554 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.280364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.281105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.281105 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.292609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1370/2728] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:04.297749 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.300260 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.306955 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:04.308812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.315736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fc0b5-427c-4864-8e03-9e526a1ec22e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.320763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1359/2729] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:04.321754 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.322120 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.322929 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.323299 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.323757 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] PATH_INFO: `/v3/projects/05d9b21fdfe34a5c9aef2def9d9ca569/users/5498f30dbfdd457eb49fedf6f09df164/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.335952 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c248e1b-8078-4fd3-aaa7-b00c2814f23c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.345030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d9b21fdfe34a5c9aef2def9d9ca569, user_id=5498f30dbfdd457eb49fedf6f09df164, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.347808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.350233 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.350987 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.351342 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.351648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.352492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:41:04.352802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.356144 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.372740 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1360/2730] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:41:04.377229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.378966 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1371/2731] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:04 2022] PUT /identity/v3/projects/05d9b21fdfe34a5c9aef2def9d9ca569/users/5498f30dbfdd457eb49fedf6f09df164/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:04.383035 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.397977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8288-3ba9-4f16-a33b-527c365bd7dc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.468696 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe8bd-dad0-429a-97c5-b29a2fa2baaa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.473982 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.474153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.474594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.474807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.474921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.475121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.475240 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.475406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.475888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.476119 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.477498 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.478065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.489238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1372/2732] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:04.493257 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.503368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600fa3-ee20-4c2b-91ac-a3e28a7a811e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.505917 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1361/2733] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:04.509219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b02b6-68a0-490e-97eb-55fc771c04ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.509219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b02b6-68a0-490e-97eb-55fc771c04ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.509414 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b02b6-68a0-490e-97eb-55fc771c04ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.509584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b02b6-68a0-490e-97eb-55fc771c04ff admin admin] PATH_INFO: `/v3/projects/05d9b21fdfe34a5c9aef2def9d9ca569/users/5498f30dbfdd457eb49fedf6f09df164/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.510000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.511976 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b02b6-68a0-490e-97eb-55fc771c04ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d9b21fdfe34a5c9aef2def9d9ca569, user_id=5498f30dbfdd457eb49fedf6f09df164, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.514881 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b02b6-68a0-490e-97eb-55fc771c04ff admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.528687 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1373/2734] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:04 2022] PUT /identity/v3/projects/05d9b21fdfe34a5c9aef2def9d9ca569/users/5498f30dbfdd457eb49fedf6f09df164/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:04.540047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.540335 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.540609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.567805 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] MFA Rules not processed for user `5498f30dbfdd457eb49fedf6f09df164`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:04.569030 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.569157 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:04.574909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505a3f7-1886-4bb5-88f2-fae1b5f07212 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.637534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.638525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.638848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.639169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.639884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.641884 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.645213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1374/2735] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:04.661188 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.662736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.664649 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1362/2736] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:41:04.668940 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.680218 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161417e-fc49-4956-8919-e68fcfac8fce admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.686495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.687217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.687505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.687785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.688522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.690131 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.703844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:04.706403 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1363/2737] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:41:04.710509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.722721 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956cfc1-3ed5-4138-a14f-1751eda3c1df admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.730037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.731194 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.731709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.732236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.733621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.735544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720afd0-5d99-435f-939d-c192a3e9aac3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.736559 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.740898 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d4f18-5e63-4f9e-bbe3-bc4a429d046e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.741565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d4f18-5e63-4f9e-bbe3-bc4a429d046e service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.741878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d4f18-5e63-4f9e-bbe3-bc4a429d046e service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.742604 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d4f18-5e63-4f9e-bbe3-bc4a429d046e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.752147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1364/2738] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:41:04.752989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d4f18-5e63-4f9e-bbe3-bc4a429d046e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.753158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.754457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d4f18-5e63-4f9e-bbe3-bc4a429d046e service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.764116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e18b2-2041-4e29-949b-d9cb2346346b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.766015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1375/2739] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:41:04.769283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.770192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.770474 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.770751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] PATH_INFO: `/v3/projects/87a25c6d9fd24da4b429a45236992831/users/96258e3c968f4dadbeaf73768cc76449/roles/e59097a97ec144ff8463625775da1081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.787033 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a25c6d9fd24da4b429a45236992831, user_id=96258e3c968f4dadbeaf73768cc76449, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.789818 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.811585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1365/2740] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:04 2022] PUT /identity/v3/projects/87a25c6d9fd24da4b429a45236992831/users/96258e3c968f4dadbeaf73768cc76449/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:04.823828 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.824050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.824326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.857468 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] MFA Rules not processed for user `96258e3c968f4dadbeaf73768cc76449`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:04.859045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.895527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.940228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1376/2741] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:04.951690 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:04.953355 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.957584 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfba85-9189-4a54-b6f7-9b01b4550339 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.962222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.962887 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.963366 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.963651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.965719 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:04.965839 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aae0c8-019c-48bd-a794-f8daf0bbd77e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:04.969471 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:04.970115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:04.970401 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:04.970681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:04.980435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:04.981862 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:04.991171 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1377/2742] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:41:05.016544 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:05.031333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:05.032855 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95162aa1-f18e-4c52-a192-1a69b7b2e1fa service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:05.037483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:05.043343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1366/2743] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:41:05.090362 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe62cd-be78-4b89-8fca-4238ca1b7996 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:05.095263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:05.096143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.096427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.096705 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] PATH_INFO: `/v3/users/96258e3c968f4dadbeaf73768cc76449` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.097906 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96258e3c968f4dadbeaf73768cc76449)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:05.099904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:05.164190 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96258e3c968f4dadbeaf73768cc76449'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.165840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:05.175917 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96258e3c968f4dadbeaf73768cc76449'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.189978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96258e3c968f4dadbeaf73768cc76449'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.190727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96258e3c968f4dadbeaf73768cc76449'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.198646 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96258e3c968f4dadbeaf73768cc76449'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.204905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1378/2744] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:05 2022] DELETE /identity/v3/users/96258e3c968f4dadbeaf73768cc76449 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:05.209521 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb23d5-c836-461d-8068-ef3c18603b28 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.209809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb23d5-c836-461d-8068-ef3c18603b28 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.210083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb23d5-c836-461d-8068-ef3c18603b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.230412 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-42eb23d5-c836-461d-8068-ef3c18603b28 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:05.231710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-42eb23d5-c836-461d-8068-ef3c18603b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:05.331192 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1367/2745] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:05.338275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:05.341806 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:05.413959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca388886-39d3-4831-a24c-4a673ef3c0e7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:05.418863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:05.419684 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.420119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.420394 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.435247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:05.436942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:05.467582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1379/2746] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:05.532598 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9089f3-c0b3-43aa-92da-6d753235eb39 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.532822 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9089f3-c0b3-43aa-92da-6d753235eb39 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.533067 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9089f3-c0b3-43aa-92da-6d753235eb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.551774 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0c9089f3-c0b3-43aa-92da-6d753235eb39 None None] MFA Rules not processed for user `00304ab3aada47ccb0756d40b5484ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:05.553010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0c9089f3-c0b3-43aa-92da-6d753235eb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:05.616903 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1368/2747] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:05.752495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:05.754219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:05.772786 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a153619-7eec-4cc6-9749-b5dcf128e2c6 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:05.777760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:05.778473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.778833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.779021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] PATH_INFO: `/v3/projects/87a25c6d9fd24da4b429a45236992831` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.780337 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a25c6d9fd24da4b429a45236992831)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:05.782287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:05.833400 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a25c6d9fd24da4b429a45236992831', 'request_id': 'req-5a1a0e78-4800-4570-91a5-41ee79fc6b15'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.840456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a25c6d9fd24da4b429a45236992831', 'request_id': 'req-5a1a0e78-4800-4570-91a5-41ee79fc6b15'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.846508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a25c6d9fd24da4b429a45236992831', 'request_id': 'req-5a1a0e78-4800-4570-91a5-41ee79fc6b15'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.861392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] The token cache is being invalidate because project 87a25c6d9fd24da4b429a45236992831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:05.861692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.862328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a1a0e78-4800-4570-91a5-41ee79fc6b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:05.863972 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1380/2748] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:05 2022] DELETE /identity/v3/projects/87a25c6d9fd24da4b429a45236992831 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:05.868837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.869132 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.869420 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.889086 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:05.890328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:05.952422 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1369/2749] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:05.960258 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:05.960566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:05.960834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:05.983686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:05.985296 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:05.993268 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1381/2750] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:05.997555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.007727 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773f09e-7ed2-4150-8512-7445e0d4b7b6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.013564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.014236 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.014519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.014802 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.015572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:41:06.015885 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.017519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.031717 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1370/2751] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:41:06.036561 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.046349 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104390e-0ed9-494f-8caa-a4f75cc5bd08 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.051075 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.051778 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.052065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.052341 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.053119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.055044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.081873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1382/2752] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:06 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:06.086107 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.097113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf5b8-2bb8-4ce0-8c15-e13921b7e606 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.135392 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:06.172383 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.173020 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.173306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.173587 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.174302 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.176428 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.207095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1371/2753] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:06 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:06.209423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.220978 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3195766-d0c1-4605-8fa3-2119ab93dc59 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.226245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.226925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.227244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.227519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.228249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.229961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.241210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1383/2754] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:06.245438 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.256573 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6b661-4a20-4b17-9182-ee060720d252 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.263623 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f666c5a-3a0a-4cc6-b48c-5bf37ddb0c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.264554 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f666c5a-3a0a-4cc6-b48c-5bf37ddb0c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.264851 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f666c5a-3a0a-4cc6-b48c-5bf37ddb0c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.265154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f666c5a-3a0a-4cc6-b48c-5bf37ddb0c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.265892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f666c5a-3a0a-4cc6-b48c-5bf37ddb0c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.267547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f666c5a-3a0a-4cc6-b48c-5bf37ddb0c59 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.283140 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1372/2755] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:06.284423 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.294913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b488d4-db6d-46a1-8fd0-632d1d8cfb4d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.299677 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.300557 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.300841 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.301218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] PATH_INFO: `/v3/projects/69c3434a76c647f99f50778ec26ffd70/users/36dcb01083d64210a3ddf86e0bda4287/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.313154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:06.322549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c3434a76c647f99f50778ec26ffd70, user_id=36dcb01083d64210a3ddf86e0bda4287, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.328034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.342001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1384/2756] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:06 2022] PUT /identity/v3/projects/69c3434a76c647f99f50778ec26ffd70/users/36dcb01083d64210a3ddf86e0bda4287/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:06.347351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.347650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.347935 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.369455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:06.371267 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:06.441957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1373/2757] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:06.457551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:06.459455 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:06.531981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2277012-f1cc-4f1e-89cb-07dad5f5676a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:06.536221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029d3da-9729-4db1-8377-c09ce308598a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:06.536912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029d3da-9729-4db1-8377-c09ce308598a service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:06.537200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029d3da-9729-4db1-8377-c09ce308598a service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:06.537482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029d3da-9729-4db1-8377-c09ce308598a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:06.551482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029d3da-9729-4db1-8377-c09ce308598a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:06.552943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029d3da-9729-4db1-8377-c09ce308598a service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:06.564085 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1385/2758] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:07.724491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:07.726186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:07.743606 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d77636-c0f7-4d72-a252-3e252d68fa5e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:07.749561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:07.750558 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:07.751039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:07.751560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] PATH_INFO: `/v3/users/620fb117ea594c6d893411302e00ab78` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:07.753202 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] RBAC: Authorizing `identity:delete_user(user_id=620fb117ea594c6d893411302e00ab78)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:07.755888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:07.814508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:07.815127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:07.815127 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:07.855382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] MFA Rules not processed for user `36dcb01083d64210a3ddf86e0bda4287`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:07.857023 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:07.861932 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620fb117ea594c6d893411302e00ab78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:07.874006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620fb117ea594c6d893411302e00ab78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:07.875032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620fb117ea594c6d893411302e00ab78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:07.894301 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620fb117ea594c6d893411302e00ab78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:07.932030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a58cdab9-7a45-4403-b1b1-b2014faad28a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620fb117ea594c6d893411302e00ab78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:07.945986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1374/2759] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:07 2022] DELETE /identity/v3/users/620fb117ea594c6d893411302e00ab78 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:07.951650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:07.955339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:07.955339 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:07.965167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1386/2760] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:07.978056 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:07.980158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:07.992731 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:07.994659 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:08.032763 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:08.063041 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55796a19-73d1-486b-944c-c5d94dcb45a1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:08.102662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1375/2761] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:08.106549 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:08.113113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:08.117621 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:08.128054 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:08.128797 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:08.129145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:08.129508 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:08.192541 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad4f8a-b0e2-49b9-8dd4-ac2d0076528e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:08.197517 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:08.199184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:08.199409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:08.200552 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:08.200871 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:08.201328 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:08.211102 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1387/2762] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:41:08.215215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:08.216684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:08.228291 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1376/2763] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:41:08.665046 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:08.666658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:08.684332 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202faa-9890-4039-aa25-8e2803f0b12c service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:08.689679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:08.690568 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:08.690853 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:08.691733 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] PATH_INFO: `/v3/projects/6efd5cce97594e9ebbd970aac9557314` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:08.692461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6efd5cce97594e9ebbd970aac9557314)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:08.699114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:08.762634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6efd5cce97594e9ebbd970aac9557314', 'request_id': 'req-33d0f6c6-fa53-42df-a417-caf783124ee4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.769419 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6efd5cce97594e9ebbd970aac9557314', 'request_id': 'req-33d0f6c6-fa53-42df-a417-caf783124ee4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.780408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6efd5cce97594e9ebbd970aac9557314', 'request_id': 'req-33d0f6c6-fa53-42df-a417-caf783124ee4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.797208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] The token cache is being invalidate because project 6efd5cce97594e9ebbd970aac9557314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:08.797447 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.798450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.800287 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1388/2764] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:08 2022] DELETE /identity/v3/projects/6efd5cce97594e9ebbd970aac9557314 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:08.804087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:08.815739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4e96-eaee-4ba9-aadf-5f0cbe0dea07 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:08.887163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:08.888630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:08.889207 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:08.889461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] PATH_INFO: `/v3/users/5498f30dbfdd457eb49fedf6f09df164` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:08.890722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5498f30dbfdd457eb49fedf6f09df164)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:08.892701 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:08.943622 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5498f30dbfdd457eb49fedf6f09df164'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.955146 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5498f30dbfdd457eb49fedf6f09df164'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.955840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5498f30dbfdd457eb49fedf6f09df164'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.965628 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5498f30dbfdd457eb49fedf6f09df164'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.975003 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:08.984646 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0efba38-4234-4a95-9d47-2e0da3eea465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5498f30dbfdd457eb49fedf6f09df164'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:08.992619 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1377/2765] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:08 2022] DELETE /identity/v3/users/5498f30dbfdd457eb49fedf6f09df164 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:09.310719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:09.312398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:09.328482 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0f6c6-fa53-42df-a417-caf783124ee4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:09.437664 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:09.438719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.439115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.439479 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] PATH_INFO: `/v3/projects/05d9b21fdfe34a5c9aef2def9d9ca569` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.440869 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d9b21fdfe34a5c9aef2def9d9ca569)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:09.442884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:09.462125 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:09.505573 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d9b21fdfe34a5c9aef2def9d9ca569', 'request_id': 'req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:09.514111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d9b21fdfe34a5c9aef2def9d9ca569', 'request_id': 'req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:09.520612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d9b21fdfe34a5c9aef2def9d9ca569', 'request_id': 'req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:09.540327 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] The token cache is being invalidate because project 05d9b21fdfe34a5c9aef2def9d9ca569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:09.540647 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:09.541377 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ec443c37-2ddf-4ed6-b92b-e06a2501fd4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:09.544588 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1389/2766] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:09 2022] DELETE /identity/v3/projects/05d9b21fdfe34a5c9aef2def9d9ca569 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:09.549883 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31e76a-a02a-4206-9b15-e05967d6e429 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.550249 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31e76a-a02a-4206-9b15-e05967d6e429 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.550466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31e76a-a02a-4206-9b15-e05967d6e429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.573997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4a31e76a-a02a-4206-9b15-e05967d6e429 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:09.574938 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4a31e76a-a02a-4206-9b15-e05967d6e429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:09.662228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.662876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.663434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.669174 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1378/2767] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:09.679072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.679367 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.679653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.688138 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] MFA Rules not processed for user `36dcb01083d64210a3ddf86e0bda4287`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:09.689628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:09.719766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:09.721124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:09.730494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1379/2768] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:09.734229 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:09.745973 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d40484-72be-4fc7-b362-58655b8f0d5e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:09.751345 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:09.752118 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.752283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.752563 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.753340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:41:09.753588 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:09.755239 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:09.766132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1390/2769] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:09.771231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1380/2770] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:41:09.774192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:09.785563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dcef4-0534-4382-9012-9f089d024906 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:09.790743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:09.791441 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.791795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.792077 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.792933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:09.794863 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:09.828312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1391/2771] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:09 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:09.829841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:09.841433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc6490-4ea5-4099-822b-de705206acff admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:09.905323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:09.906006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.906444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.906570 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.907294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:09.909223 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:09.934603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1381/2772] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:09 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:41:09.938735 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:09.950038 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8574b4a-78a8-4af0-a435-a3cf2ebbae81 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:09.954709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:09.955454 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.955724 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.956008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.956741 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:09.958360 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:09.970706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1392/2773] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:09.974484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:09.985778 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3ef0e-956a-4cb4-a796-6b87d3c141a4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:09.991110 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe341803-25ef-42e2-ae04-13f25c6b99c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:09.991308 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341803-25ef-42e2-ae04-13f25c6b99c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:09.991597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341803-25ef-42e2-ae04-13f25c6b99c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:09.991875 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341803-25ef-42e2-ae04-13f25c6b99c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:09.992650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe341803-25ef-42e2-ae04-13f25c6b99c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:09.994255 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe341803-25ef-42e2-ae04-13f25c6b99c3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.008736 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1382/2774] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:10.009930 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.020659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff22f5-45ce-45fd-ae2b-999c30845e2f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.025392 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.026324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.026597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.026879 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] PATH_INFO: `/v3/projects/965189d926494825b026010f55270851/users/7ea7f3ad96f143f4bc688ee30079a8d9/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.042232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=965189d926494825b026010f55270851, user_id=7ea7f3ad96f143f4bc688ee30079a8d9, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.044913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.064583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1393/2775] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:10 2022] PUT /identity/v3/projects/965189d926494825b026010f55270851/users/7ea7f3ad96f143f4bc688ee30079a8d9/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:10.084069 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.084607 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.085095 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.131651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] MFA Rules not processed for user `7ea7f3ad96f143f4bc688ee30079a8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:10.132893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:10.168557 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:10.206273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1383/2776] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:10.210729 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.274216 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:10.288291 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8586f4-381e-4ee1-b114-35a321b73d6d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.294657 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.295375 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.295666 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.295946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.296656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.298613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.326339 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1394/2777] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:10 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:10.330674 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.343310 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9678a-b08f-49ac-a355-4dce67689e7d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.428960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.429867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.430148 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.430543 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.431519 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.433745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.456186 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1384/2778] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:10 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:10.461596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.473187 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e5296-c0ba-436c-be71-7f2abce9d27d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.481474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.482177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.482459 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.482899 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.483675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.486235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.498013 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1395/2779] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:10.501812 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.513658 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154adf8-1110-457a-9e68-48d775b2095b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.518641 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff503-aa0f-4b38-a6ea-b9801fa79a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.519331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff503-aa0f-4b38-a6ea-b9801fa79a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.519614 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff503-aa0f-4b38-a6ea-b9801fa79a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.519888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff503-aa0f-4b38-a6ea-b9801fa79a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.520655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff503-aa0f-4b38-a6ea-b9801fa79a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.522381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff503-aa0f-4b38-a6ea-b9801fa79a53 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.534344 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1385/2780] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:10.539049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.556523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f73a-c599-45e1-a698-f676dc854513 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.564457 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.566230 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.566551 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.566892 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] PATH_INFO: `/v3/projects/74eaaa8b667e4234a2c5c1fef9b3eec3/users/83e0e6d081c841f69ba6a2fa7fba9a17/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.592112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74eaaa8b667e4234a2c5c1fef9b3eec3, user_id=83e0e6d081c841f69ba6a2fa7fba9a17, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.596179 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.622970 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1396/2781] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:10 2022] PUT /identity/v3/projects/74eaaa8b667e4234a2c5c1fef9b3eec3/users/83e0e6d081c841f69ba6a2fa7fba9a17/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:10.634961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.635321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.635780 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.676985 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] MFA Rules not processed for user `83e0e6d081c841f69ba6a2fa7fba9a17`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:10.677971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:10.783603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1386/2782] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:10.788345 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.871529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dbfe7-e796-4d71-86d7-f4597f6451b5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:10.880181 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:10.881867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:10.882164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:10.882639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:10.883404 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:10.886692 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:10.936055 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1397/2783] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:10 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:10.940434 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:10.953170 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eec1e-d42b-4bc6-845f-c79c03f2cd00 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.003399 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:11.025376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.027026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.027900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.028246 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.029085 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.031088 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.059767 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1387/2784] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:10 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:41:11.063347 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:11.074920 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98d639-0d98-4119-9434-65ee51dd9a71 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.080676 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.081439 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.081753 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.082025 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.082767 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.084487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.098126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1398/2785] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:11.102361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:11.112792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06474a03-0373-4123-9671-415ca5f15b05 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.117709 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.118597 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.118877 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.119185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] PATH_INFO: `/v3/projects/84d8754b4fdd4faabc913fb23881abd2/users/e1e06f0948ba4ebea9b371a38163e9aa/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.136427 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d8754b4fdd4faabc913fb23881abd2, user_id=e1e06f0948ba4ebea9b371a38163e9aa, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.139497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.153776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1388/2786] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:11 2022] PUT /identity/v3/projects/84d8754b4fdd4faabc913fb23881abd2/users/e1e06f0948ba4ebea9b371a38163e9aa/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:11.157961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:11.168097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459927-b8b3-481b-8e7f-e0bfce3458fd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.226047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795062fe-49c5-4829-a4a1-95ad7b058009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.226728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795062fe-49c5-4829-a4a1-95ad7b058009 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.227016 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795062fe-49c5-4829-a4a1-95ad7b058009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.227328 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795062fe-49c5-4829-a4a1-95ad7b058009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.228065 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795062fe-49c5-4829-a4a1-95ad7b058009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.230033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795062fe-49c5-4829-a4a1-95ad7b058009 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.241050 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1399/2787] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:11.246007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:11.257072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c23d2-f1c2-4713-973a-0275d250a22f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.262457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.263662 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.263941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.264219 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] PATH_INFO: `/v3/projects/84d8754b4fdd4faabc913fb23881abd2/users/e1e06f0948ba4ebea9b371a38163e9aa/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.266221 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d8754b4fdd4faabc913fb23881abd2, user_id=e1e06f0948ba4ebea9b371a38163e9aa, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.268923 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.285909 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1389/2788] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:11 2022] PUT /identity/v3/projects/84d8754b4fdd4faabc913fb23881abd2/users/e1e06f0948ba4ebea9b371a38163e9aa/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:11.296925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.297221 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.297516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.318166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:11.330563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] MFA Rules not processed for user `e1e06f0948ba4ebea9b371a38163e9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:11.331874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:11.421986 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1400/2789] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:11.435283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:11.437075 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:11.560002 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409702c-1de7-4508-b56d-b13147770009 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.563814 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.564680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.565032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.565495 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.576436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.577975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.589946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1390/2790] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:11.633891 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:11.636409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:11.650132 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703529-62d2-4dec-92ef-7fb27f98735c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:11.655151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee0780-e963-425e-8fbe-fa4268bdaa23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:11.656609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee0780-e963-425e-8fbe-fa4268bdaa23 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:11.656609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee0780-e963-425e-8fbe-fa4268bdaa23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:11.656609 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee0780-e963-425e-8fbe-fa4268bdaa23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:11.726842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee0780-e963-425e-8fbe-fa4268bdaa23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:11.728370 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee0780-e963-425e-8fbe-fa4268bdaa23 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:11.739989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1401/2791] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:14.664706 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:14.729831 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52879ad-ae8a-446a-81cc-e7c19f5a9077 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:14.734461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:14.735376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:14.735791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:14.736164 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] PATH_INFO: `/v3/users/7ea7f3ad96f143f4bc688ee30079a8d9` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:14.737374 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea7f3ad96f143f4bc688ee30079a8d9)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:14.739346 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:14.762760 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:14.787755 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea7f3ad96f143f4bc688ee30079a8d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:14.795470 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea7f3ad96f143f4bc688ee30079a8d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:14.796101 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea7f3ad96f143f4bc688ee30079a8d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:14.803307 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea7f3ad96f143f4bc688ee30079a8d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:14.815858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea7f3ad96f143f4bc688ee30079a8d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:14.822211 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1391/2792] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:14 2022] DELETE /identity/v3/users/7ea7f3ad96f143f4bc688ee30079a8d9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:14.826852 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:14.827367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:14.827485 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:14.846503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:14.848463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:14.912912 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1402/2793] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:14.923437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:14.925082 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:14.997599 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0de322-1dfe-4f36-b35d-9acf31e024c6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:15.001565 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:15.002240 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:15.002524 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:15.002805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:15.013351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:15.014803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:15.024864 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1392/2794] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:15.303720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:15.305160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:15.318154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081290d-9bc7-4759-a6e4-dfa34b59b83a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:15.323650 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:15.324631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:15.324933 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:15.325213 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] PATH_INFO: `/v3/projects/965189d926494825b026010f55270851` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:15.326443 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=965189d926494825b026010f55270851)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:15.328645 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:15.365617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:15.379622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965189d926494825b026010f55270851', 'request_id': 'req-4fd324db-4f22-4805-91bd-87c153c0e8c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.386584 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965189d926494825b026010f55270851', 'request_id': 'req-4fd324db-4f22-4805-91bd-87c153c0e8c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.392491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965189d926494825b026010f55270851', 'request_id': 'req-4fd324db-4f22-4805-91bd-87c153c0e8c7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.405628 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] The token cache is being invalidate because project 965189d926494825b026010f55270851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:15.405938 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.406574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.408166 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1403/2795] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:15 2022] DELETE /identity/v3/projects/965189d926494825b026010f55270851 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:15.412147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:15.422251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc4b2f-5d9b-4fe2-ace3-6857b99f74a6 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:15.478996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:15.480047 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:15.480323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:15.480596 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] PATH_INFO: `/v3/users/83e0e6d081c841f69ba6a2fa7fba9a17` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:15.481793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e0e6d081c841f69ba6a2fa7fba9a17)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:15.483777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:15.528873 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:15.532136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e0e6d081c841f69ba6a2fa7fba9a17'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.540753 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e0e6d081c841f69ba6a2fa7fba9a17'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.541879 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e0e6d081c841f69ba6a2fa7fba9a17'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.549534 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e0e6d081c841f69ba6a2fa7fba9a17'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.562279 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e0e6d081c841f69ba6a2fa7fba9a17'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:15.568343 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1393/2796] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:15 2022] DELETE /identity/v3/users/83e0e6d081c841f69ba6a2fa7fba9a17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:15.857378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:15.858842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:15.879574 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd324db-4f22-4805-91bd-87c153c0e8c7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:15.941427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:15.942597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:15.942993 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:15.943550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] PATH_INFO: `/v3/projects/74eaaa8b667e4234a2c5c1fef9b3eec3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:15.944989 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74eaaa8b667e4234a2c5c1fef9b3eec3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:15.947024 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:16.001883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74eaaa8b667e4234a2c5c1fef9b3eec3', 'request_id': 'req-36045e53-09af-4b10-af68-2d1184cb92c5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.008196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74eaaa8b667e4234a2c5c1fef9b3eec3', 'request_id': 'req-36045e53-09af-4b10-af68-2d1184cb92c5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.016232 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74eaaa8b667e4234a2c5c1fef9b3eec3', 'request_id': 'req-36045e53-09af-4b10-af68-2d1184cb92c5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.030263 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] The token cache is being invalidate because project 74eaaa8b667e4234a2c5c1fef9b3eec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:16.030550 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.031244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.032795 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1404/2797] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:15 2022] DELETE /identity/v3/projects/74eaaa8b667e4234a2c5c1fef9b3eec3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:16.036867 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:16.038390 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:16.053914 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ea4c-62dd-4157-b55d-988d3f0c5ee3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:16.124553 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:16.125455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:16.125741 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:16.126055 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] PATH_INFO: `/v3/users/e1e06f0948ba4ebea9b371a38163e9aa` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:16.127509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e06f0948ba4ebea9b371a38163e9aa)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:16.129509 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:16.197687 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e06f0948ba4ebea9b371a38163e9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.206782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e06f0948ba4ebea9b371a38163e9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.207988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e06f0948ba4ebea9b371a38163e9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.216957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e06f0948ba4ebea9b371a38163e9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.233858 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-73365336-641b-49c3-b58d-f08b9a18c61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e06f0948ba4ebea9b371a38163e9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.242529 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1394/2798] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:16 2022] DELETE /identity/v3/users/e1e06f0948ba4ebea9b371a38163e9aa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:16.646545 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:16.648044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:16.662336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36045e53-09af-4b10-af68-2d1184cb92c5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:16.725218 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:16.738621 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:16.739531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:16.739819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:16.740122 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] PATH_INFO: `/v3/projects/84d8754b4fdd4faabc913fb23881abd2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:16.741366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d8754b4fdd4faabc913fb23881abd2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:16.743326 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:16.798321 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d8754b4fdd4faabc913fb23881abd2', 'request_id': 'req-71cc577e-3498-4bd8-8662-ac34f8c3d529'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.806503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d8754b4fdd4faabc913fb23881abd2', 'request_id': 'req-71cc577e-3498-4bd8-8662-ac34f8c3d529'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.812968 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d8754b4fdd4faabc913fb23881abd2', 'request_id': 'req-71cc577e-3498-4bd8-8662-ac34f8c3d529'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.828683 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] The token cache is being invalidate because project 84d8754b4fdd4faabc913fb23881abd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:16.828981 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.829642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:16.831261 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1405/2799] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:16 2022] DELETE /identity/v3/projects/84d8754b4fdd4faabc913fb23881abd2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:21.791910 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134fcfd-7492-4616-9ac2-59740e47a992 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:21.793017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134fcfd-7492-4616-9ac2-59740e47a992 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:21.793160 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134fcfd-7492-4616-9ac2-59740e47a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:21.810944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f134fcfd-7492-4616-9ac2-59740e47a992 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:21.812213 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f134fcfd-7492-4616-9ac2-59740e47a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:21.850695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:21.879147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1395/2800] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:41:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:21.892878 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:21.894422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:21.972600 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc577e-3498-4bd8-8662-ac34f8c3d529 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:21.982874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:21.983759 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:21.984634 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:21.984923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:21.995247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:21.996910 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:22.009495 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1406/2801] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:41:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:41:22.567497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30c961-0e30-4948-9be8-6e5b0d616178 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:22.567793 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30c961-0e30-4948-9be8-6e5b0d616178 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:22.568074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30c961-0e30-4948-9be8-6e5b0d616178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:22.587473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4c30c961-0e30-4948-9be8-6e5b0d616178 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:22.588700 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4c30c961-0e30-4948-9be8-6e5b0d616178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:22.596523 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1396/2802] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jul 21 22:41:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:22.609432 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:22.620115 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189ba9-6512-48c1-9236-2b9340f08103 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:22.625435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:22.626097 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:22.626383 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:22.626688 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:22.635469 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:22.637803 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:22.646158 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1407/2803] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:41:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 22:41:22.845478 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d0c69-23c5-4378-8ca5-9ac5c659ea10 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:22.846319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d0c69-23c5-4378-8ca5-9ac5c659ea10 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:22.846611 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d0c69-23c5-4378-8ca5-9ac5c659ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:22.867112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f72d0c69-23c5-4378-8ca5-9ac5c659ea10 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:22.868347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f72d0c69-23c5-4378-8ca5-9ac5c659ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:22.876043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1397/2804] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jul 21 22:41:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:41:22.888927 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:22.900034 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0689f-d784-44c7-ac7b-fb9734549d19 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:22.906151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:22.906830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:22.907118 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:22.907531 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:22.916147 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:22.917960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:22.926167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1408/2805] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:41:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:41:23.269766 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:23.270058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:23.270340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:23.288892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] MFA Rules not processed for user `48141af292e144caa3ea38613964403e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:23.290125 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:23.297805 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1398/2806] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jul 21 22:41:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:23.310810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:23.326504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6051a27-7c5e-4a5f-a6fc-5768c2a210b5 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:23.331777 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e55b3c-6b60-4733-bbcb-f601a3407a30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08ed51a3aab34ac0bf5791fa3307a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:23.332440 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e55b3c-6b60-4733-bbcb-f601a3407a30 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:23.332727 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e55b3c-6b60-4733-bbcb-f601a3407a30 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:23.333007 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e55b3c-6b60-4733-bbcb-f601a3407a30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:23.342160 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e55b3c-6b60-4733-bbcb-f601a3407a30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:23.343808 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e55b3c-6b60-4733-bbcb-f601a3407a30 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:23.352742 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1409/2807] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jul 21 22:41:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:41:25.465243 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:25.466799 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:25.557724 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83677d-ae38-4365-9017-d71a0c8e5150 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:25.563512 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:25.564446 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:25.564729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:25.565006 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] PATH_INFO: `/v3/users/00304ab3aada47ccb0756d40b5484ea3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:25.566245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00304ab3aada47ccb0756d40b5484ea3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:25.568254 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:25.627650 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00304ab3aada47ccb0756d40b5484ea3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:25.636653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00304ab3aada47ccb0756d40b5484ea3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:25.637590 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00304ab3aada47ccb0756d40b5484ea3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:25.645158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00304ab3aada47ccb0756d40b5484ea3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:25.659897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00304ab3aada47ccb0756d40b5484ea3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:25.666070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1399/2808] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:41:25 2022] DELETE /identity/v3/users/00304ab3aada47ccb0756d40b5484ea3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:41:25.670715 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:25.671259 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:25.671369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:25.690140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:25.691598 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:25.719610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:25.756566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1410/2809] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:25.767624 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:25.769066 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:25.818349 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:25.832340 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1e60a-1f8c-42f3-b916-afc7ed00da93 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:25.836452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ef73-5c0a-46df-8ba8-9da9a276854c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:25.837147 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ef73-5c0a-46df-8ba8-9da9a276854c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:25.837432 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ef73-5c0a-46df-8ba8-9da9a276854c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:25.837711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ef73-5c0a-46df-8ba8-9da9a276854c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:25.848993 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ef73-5c0a-46df-8ba8-9da9a276854c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:25.850464 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ef73-5c0a-46df-8ba8-9da9a276854c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:25.864153 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1400/2810] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:26.044382 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.045940 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:26.061567 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f549ce-9d5a-471f-bfaf-e41d4f87cd7a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.066488 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.067473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.067763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.068047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] PATH_INFO: `/v3/projects/4d1af09e3e2b40be92caf9c0ef51b099` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.069252 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1af09e3e2b40be92caf9c0ef51b099)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.071306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.122747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1af09e3e2b40be92caf9c0ef51b099', 'request_id': 'req-8508b11f-b162-4b70-9360-505e38463d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:26.128998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1af09e3e2b40be92caf9c0ef51b099', 'request_id': 'req-8508b11f-b162-4b70-9360-505e38463d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:26.134505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1af09e3e2b40be92caf9c0ef51b099', 'request_id': 'req-8508b11f-b162-4b70-9360-505e38463d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:26.149251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] The token cache is being invalidate because project 4d1af09e3e2b40be92caf9c0ef51b099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:41:26.149543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:26.150183 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8508b11f-b162-4b70-9360-505e38463d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:41:26.152725 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1411/2811] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:41:26 2022] DELETE /identity/v3/projects/4d1af09e3e2b40be92caf9c0ef51b099 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:26.157668 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.157961 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.158242 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.177477 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:26.178678 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:26.250415 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1401/2812] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:26.258473 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.258760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.259044 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.280934 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:26.282148 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:26.290301 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1412/2813] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:26.296378 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.306331 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574c3d2-178e-4e13-96b9-eee80317ab4c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.311785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.312454 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.312743 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.313791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.314644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:41:26.314996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.316711 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.329896 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1402/2814] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:41:26.334577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.344825 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091ea4-bf98-432e-9c85-700a308c9773 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.349523 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.350200 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.350481 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.350745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.351492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.353390 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.384942 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1413/2815] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:26 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:41:26.386729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.397204 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eafea7-7851-40d8-b410-11e4f9d70aae admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.458166 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.459039 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.459394 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.459681 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.460397 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.462463 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.484920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1403/2816] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:26 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:41:26.489191 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.500219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f21b5d-38fa-4d0e-ade8-e8aca3f531ed admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.505320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.505963 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.506249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.506529 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.507306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.508958 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.521649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1414/2817] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:41:26.525547 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.537090 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b43e19-1a64-45c7-b7f6-7436cf244d62 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.541807 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25e43-c1ff-4dbe-9f99-88123f743db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.542466 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25e43-c1ff-4dbe-9f99-88123f743db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.542756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25e43-c1ff-4dbe-9f99-88123f743db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.543205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25e43-c1ff-4dbe-9f99-88123f743db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.543943 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25e43-c1ff-4dbe-9f99-88123f743db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.545572 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25e43-c1ff-4dbe-9f99-88123f743db2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.556320 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1404/2818] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:26.561006 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.571015 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314e9f3-d02c-494d-bc60-f4b7d3feeb08 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.576145 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.577021 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.577297 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.577570 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] PATH_INFO: `/v3/projects/e61e273f945e45f0a6e1f3b45e4bc9d2/users/a367906e74654507aaed8b885d525a0f/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.592923 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61e273f945e45f0a6e1f3b45e4bc9d2, user_id=a367906e74654507aaed8b885d525a0f, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.595644 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.609818 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1415/2819] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:26 2022] PUT /identity/v3/projects/e61e273f945e45f0a6e1f3b45e4bc9d2/users/a367906e74654507aaed8b885d525a0f/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:26.615280 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925aeae-ccbf-447f-bbd2-050233b38edd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.615576 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925aeae-ccbf-447f-bbd2-050233b38edd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.615925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925aeae-ccbf-447f-bbd2-050233b38edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.634108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7925aeae-ccbf-447f-bbd2-050233b38edd None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:26.635360 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7925aeae-ccbf-447f-bbd2-050233b38edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:26.692825 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:26.708057 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1405/2820] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:26.718582 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:26.720496 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:26.781198 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:26.786842 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1422f35-5d0a-4379-a34d-5d513406681b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:26.791998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:26.792859 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:26.793244 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:26.794101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:26.809215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:26.811189 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:26.822112 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1416/2821] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:31.190502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.191026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.191026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.234632 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] MFA Rules not processed for user `a367906e74654507aaed8b885d525a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:31.235924 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:31.312348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1406/2822] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:31.317121 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:31.330941 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c867fb-bf95-4b19-bcf3-314d28d8817e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:31.336089 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:31.336752 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.337036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.337510 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.338210 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:31.341312 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:31.375368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1417/2823] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:41:31 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:41:31.376496 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:31.387892 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45086e-144b-446c-8a0a-f82f89c199df None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:31.458431 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:31.459257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.459560 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.459841 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.460583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:31.462602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:31.484257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1407/2824] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:41:31 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:41:31.488410 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:31.500139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9a88c-4079-4df8-8b4c-fe0d29982fc2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:31.506830 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:31.507663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.508047 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.508422 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.509249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:31.512157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:31.523807 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1418/2825] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:31.533502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:31.545076 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f44e16-10fa-47c9-a97b-4d9b81dd96ce admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:31.549783 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:31.550739 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.551026 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.551325 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] PATH_INFO: `/v3/projects/1fe0b11f114e440582f0edbe0614caa9/users/46bce9df7ec449389efe4aa68ecd9b03/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.571283 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe0b11f114e440582f0edbe0614caa9, user_id=46bce9df7ec449389efe4aa68ecd9b03, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:31.574929 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:31.596369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1408/2826] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:31 2022] PUT /identity/v3/projects/1fe0b11f114e440582f0edbe0614caa9/users/46bce9df7ec449389efe4aa68ecd9b03/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:31.601760 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:31.612149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af522e80-2051-4288-a903-39fbd447b623 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:31.670225 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045d932-71e7-465d-af43-98b8d70bc1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:31.670908 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045d932-71e7-465d-af43-98b8d70bc1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.671228 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045d932-71e7-465d-af43-98b8d70bc1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.671512 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045d932-71e7-465d-af43-98b8d70bc1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.672249 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045d932-71e7-465d-af43-98b8d70bc1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:31.673865 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045d932-71e7-465d-af43-98b8d70bc1d0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:31.685763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1419/2827] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:41:31.689878 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:31.700920 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea36f1-8514-41c9-be16-e5e10451b98b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:31.705457 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:31.706369 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:31.706661 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:31.706941 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] PATH_INFO: `/v3/projects/1fe0b11f114e440582f0edbe0614caa9/users/46bce9df7ec449389efe4aa68ecd9b03/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:31.709007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe0b11f114e440582f0edbe0614caa9, user_id=46bce9df7ec449389efe4aa68ecd9b03, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:31.712010 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:31.723515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1409/2828] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:41:31 2022] PUT /identity/v3/projects/1fe0b11f114e440582f0edbe0614caa9/users/46bce9df7ec449389efe4aa68ecd9b03/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:41:37.012031 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dfb8-1d4b-422b-8afa-9bb793ecca6a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:37.012612 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dfb8-1d4b-422b-8afa-9bb793ecca6a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:37.012819 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dfb8-1d4b-422b-8afa-9bb793ecca6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:37.052247 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9920dfb8-1d4b-422b-8afa-9bb793ecca6a None None] MFA Rules not processed for user `46bce9df7ec449389efe4aa68ecd9b03`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:37.053577 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9920dfb8-1d4b-422b-8afa-9bb793ecca6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:37.145149 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1420/2829] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:41:37.162644 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:41:37.165165 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:37.211251 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:37.272582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d7b8a-bfd8-4df8-aceb-1ce6bb415109 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:41:37.279765 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f20c72-426d-48d9-904a-1662479e7a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:41:37.280433 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f20c72-426d-48d9-904a-1662479e7a80 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:37.280752 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f20c72-426d-48d9-904a-1662479e7a80 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:37.281030 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f20c72-426d-48d9-904a-1662479e7a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:37.340730 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f20c72-426d-48d9-904a-1662479e7a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:41:37.342348 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f20c72-426d-48d9-904a-1662479e7a80 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:41:37.352032 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1410/2830] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:41:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:41:47.141682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:47.347444 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=49 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:41:57.910126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:41:57.910701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:41:57.910701 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:41:57.928823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] MFA Rules not processed for user `a367906e74654507aaed8b885d525a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:41:57.929959 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:41:57.936905 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1421/2831] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:41:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:42:07.923110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:42:13.230269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:42:13.230823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:42:13.230823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:42:13.248996 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] MFA Rules not processed for user `36dcb01083d64210a3ddf86e0bda4287`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:42:13.250975 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:13.321193 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1411/2832] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:42:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:42:16.583403 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:42:16.584998 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:16.651269 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1ace-392e-4ec8-918f-159caee44c6d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:42:16.655860 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:42:16.657430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:42:16.657787 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:42:16.658070 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:42:16.671513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:42:16.672619 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:42:16.683452 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1422/2833] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:42:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:42:18.735776 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:42:18.738064 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:18.755608 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250da1a9-10be-4d47-ada8-09cb06da72cf None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:42:18.761461 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:42:18.762197 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:42:18.762555 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:42:18.762906 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:42:18.765133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:18.779074 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:42:18.780670 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:42:18.792809 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1412/2834] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:42:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:42:26.678728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=26 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:42:28.788893 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=31 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:08.902450 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:08.903921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:08.974928 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:08.976811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:09.000876 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ab934-2e89-44ac-925b-997fdc308c1d service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.007921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.008681 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.009088 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.009411 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.011304 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:09.024547 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.026663 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.038913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1423/2835] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:09.068389 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480dd59-5ecf-4cf4-a011-e6323af956da service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.075494 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.076583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.077143 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.077430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] PATH_INFO: `/v3/users/36dcb01083d64210a3ddf86e0bda4287` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.078751 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36dcb01083d64210a3ddf86e0bda4287)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.081368 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.126121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36dcb01083d64210a3ddf86e0bda4287'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.134690 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36dcb01083d64210a3ddf86e0bda4287'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.135324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36dcb01083d64210a3ddf86e0bda4287'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.143630 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36dcb01083d64210a3ddf86e0bda4287'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.155276 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70ce4152-0b93-450f-a374-c3bfeecf1b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36dcb01083d64210a3ddf86e0bda4287'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.161683 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1413/2836] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:43:08 2022] DELETE /identity/v3/users/36dcb01083d64210a3ddf86e0bda4287 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:43:09.336743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.350814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca221b-bf12-4d5b-948b-9ebff1c1ae07 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.412336 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.413199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.413478 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.413747 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] PATH_INFO: `/v3/projects/69c3434a76c647f99f50778ec26ffd70` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.414918 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c3434a76c647f99f50778ec26ffd70)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.416868 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.464193 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c3434a76c647f99f50778ec26ffd70', 'request_id': 'req-498f3b05-f09a-49aa-a2a3-d1ff37871069'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.470622 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c3434a76c647f99f50778ec26ffd70', 'request_id': 'req-498f3b05-f09a-49aa-a2a3-d1ff37871069'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.476943 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c3434a76c647f99f50778ec26ffd70', 'request_id': 'req-498f3b05-f09a-49aa-a2a3-d1ff37871069'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.510222 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] The token cache is being invalidate because project 69c3434a76c647f99f50778ec26ffd70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:43:09.510504 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.511201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-498f3b05-f09a-49aa-a2a3-d1ff37871069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:09.512922 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1424/2837] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:43:09 2022] DELETE /identity/v3/projects/69c3434a76c647f99f50778ec26ffd70 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:09.518257 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.518551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.518836 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.538205 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:09.539483 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:09.600958 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1414/2838] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:43:09.608887 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.612002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.612002 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.631267 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:09.632509 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:09.640745 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1425/2839] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:09.646313 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.656058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f0948-90b0-4c57-a210-424651ea177c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.661505 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.662323 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.662439 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.662722 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.663502 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:43:09.663813 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.665459 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.678083 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1415/2840] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:43:09.682363 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.692275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fa18-e0fb-4a03-8cea-014092306bb1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.697563 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.698229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.698513 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.698816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.699564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.701539 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.713131 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:09.741196 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1426/2841] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:43:09 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:43:09.745492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.756785 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e9080-0c65-4a98-b59b-bc7f6d0e09d5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.760561 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:09.822695 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.823401 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.823684 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.823956 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.824639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.826569 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.847161 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1416/2842] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:43:09 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:43:09.851493 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.862368 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46cc8b-8ab9-425d-b666-e3fead7a75a5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.867274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.867907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.868227 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.868532 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.869254 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.870873 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.882281 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1427/2843] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:43:09.886759 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.898081 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83f15e-1cca-45e8-a9af-8b6f31023610 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.902777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4e697-b1db-4a86-aeff-9c8ebe1d127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.903469 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e697-b1db-4a86-aeff-9c8ebe1d127d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.903756 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e697-b1db-4a86-aeff-9c8ebe1d127d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.904034 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e697-b1db-4a86-aeff-9c8ebe1d127d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.904792 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4e697-b1db-4a86-aeff-9c8ebe1d127d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.906377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4e697-b1db-4a86-aeff-9c8ebe1d127d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.917321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1417/2844] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:43:09.921521 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:09.931583 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9faaaa-e058-4701-aecd-182f4f781dbd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:09.936373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:09.937437 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.937728 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.938033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] PATH_INFO: `/v3/projects/13fec00ad1d34d30a135e37bea3bbba1/users/be77000e0781450da7bd1bc673fe5b26/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:09.953209 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13fec00ad1d34d30a135e37bea3bbba1, user_id=be77000e0781450da7bd1bc673fe5b26, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:09.956679 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:09.970076 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1428/2845] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:43:09 2022] PUT /identity/v3/projects/13fec00ad1d34d30a135e37bea3bbba1/users/be77000e0781450da7bd1bc673fe5b26/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:09.980206 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:09.980710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:09.980848 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:10.008208 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] MFA Rules not processed for user `be77000e0781450da7bd1bc673fe5b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:10.009361 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:10.074988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1418/2846] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:10.086516 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:10.088246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:10.144700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fe53e-615e-43ae-b0d1-5376881d988b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:10.148406 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143f4fa-e303-4f5f-ac6f-32c81f36b9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:10.149049 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f4fa-e303-4f5f-ac6f-32c81f36b9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:10.149334 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f4fa-e303-4f5f-ac6f-32c81f36b9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:10.149613 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f4fa-e303-4f5f-ac6f-32c81f36b9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:10.158610 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143f4fa-e303-4f5f-ac6f-32c81f36b9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:10.160080 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143f4fa-e303-4f5f-ac6f-32c81f36b9c0 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:10.168937 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1429/2847] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:20.075894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:20.164843 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:26.370501 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:26.435615 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a27b0-8995-430c-93ff-2e47b62165b2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:26.440396 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:26.441274 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:26.441556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:26.441828 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] PATH_INFO: `/v3/users/be77000e0781450da7bd1bc673fe5b26` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:26.443927 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be77000e0781450da7bd1bc673fe5b26)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:26.445513 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:26.492351 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be77000e0781450da7bd1bc673fe5b26'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:26.499437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be77000e0781450da7bd1bc673fe5b26'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:26.500053 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be77000e0781450da7bd1bc673fe5b26'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:26.507665 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be77000e0781450da7bd1bc673fe5b26'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:26.519515 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be77000e0781450da7bd1bc673fe5b26'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:26.525729 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1419/2848] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:43:26 2022] DELETE /identity/v3/users/be77000e0781450da7bd1bc673fe5b26 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:26.530409 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:26.530874 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:26.530994 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:26.548565 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:26.549800 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:26.611274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1430/2849] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:26.621499 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:26.622897 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:26.681827 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849ba2b-f020-4e91-bb90-9591b2b27cc0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:26.685486 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605720ba-2500-49f5-84f5-9b9bc9be5042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:26.686129 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605720ba-2500-49f5-84f5-9b9bc9be5042 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:26.686407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605720ba-2500-49f5-84f5-9b9bc9be5042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:26.686679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605720ba-2500-49f5-84f5-9b9bc9be5042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:26.696957 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605720ba-2500-49f5-84f5-9b9bc9be5042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:26.698465 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605720ba-2500-49f5-84f5-9b9bc9be5042 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:26.708667 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1420/2850] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:26.986543 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:26.988111 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:27.006913 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4477d8-643e-4933-9f6e-2e0df8a1be4b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.012840 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.013992 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.014275 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.014580 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] PATH_INFO: `/v3/projects/13fec00ad1d34d30a135e37bea3bbba1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.015823 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13fec00ad1d34d30a135e37bea3bbba1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.017798 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.068682 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13fec00ad1d34d30a135e37bea3bbba1', 'request_id': 'req-0aeb2980-a782-45d9-80e6-2278e6c15381'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:27.075366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13fec00ad1d34d30a135e37bea3bbba1', 'request_id': 'req-0aeb2980-a782-45d9-80e6-2278e6c15381'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:27.081748 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13fec00ad1d34d30a135e37bea3bbba1', 'request_id': 'req-0aeb2980-a782-45d9-80e6-2278e6c15381'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:27.095157 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] The token cache is being invalidate because project 13fec00ad1d34d30a135e37bea3bbba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:43:27.095444 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:27.096114 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aeb2980-a782-45d9-80e6-2278e6c15381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:43:27.097697 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1431/2851] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:43:26 2022] DELETE /identity/v3/projects/13fec00ad1d34d30a135e37bea3bbba1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:27.103293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.103718 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.104113 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.126152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:27.127314 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:27.145198 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:27.191108 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1421/2852] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:27.253904 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.254192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.254476 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.274616 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:27.275872 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:27.283961 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1432/2853] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:27.288826 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.298922 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32afc-e42e-48ef-b097-b7c394876a02 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.304493 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.305155 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.305442 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.305725 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.306492 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:43:27.306804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.308484 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.320888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1422/2854] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jul 21 22:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:43:27.325223 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.335028 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7d9b3-c5ce-4092-9470-e77b5b84075b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.339712 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.340364 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.340648 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.340926 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.341626 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.343564 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.369810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1433/2855] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:43:27 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:43:27.374139 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.386585 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798636-9952-402f-8d7c-ee36ab54c1a9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.446430 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.447154 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.447437 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.447713 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.448456 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.450436 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.470136 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1423/2856] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:43:27 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:43:27.474273 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.484834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80bb02-4eeb-4937-b16c-959ff36a5ca5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.490153 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.490884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.491208 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.491491 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.492226 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.493849 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.504433 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1434/2857] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:43:27.509562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.519838 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c803f0-5b2a-462a-aebc-475216d53278 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.524971 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fab505-0bf2-4474-9017-e4ea8985e556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.525647 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fab505-0bf2-4474-9017-e4ea8985e556 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.526065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fab505-0bf2-4474-9017-e4ea8985e556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.526326 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fab505-0bf2-4474-9017-e4ea8985e556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.527080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fab505-0bf2-4474-9017-e4ea8985e556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.528744 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fab505-0bf2-4474-9017-e4ea8985e556 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.539823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1424/2858] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:43:27.543888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.554620 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51f120-910f-437c-8bbc-e8d1f8b54b4a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.559250 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.560126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.560408 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.560685 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] PATH_INFO: `/v3/projects/8323020521c94131b7d9bef5aa456a70/users/84f06063489847e68831b2c94fac2a26/roles/e59097a97ec144ff8463625775da1081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.575381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8323020521c94131b7d9bef5aa456a70, user_id=84f06063489847e68831b2c94fac2a26, role_id=e59097a97ec144ff8463625775da1081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.578033 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.590890 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1435/2859] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:43:27 2022] PUT /identity/v3/projects/8323020521c94131b7d9bef5aa456a70/users/84f06063489847e68831b2c94fac2a26/roles/e59097a97ec144ff8463625775da1081 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:27.597273 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dd732-6ad4-420b-8427-72c23cfe6ed2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.597791 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dd732-6ad4-420b-8427-72c23cfe6ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.598294 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dd732-6ad4-420b-8427-72c23cfe6ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.623222 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6f5dd732-6ad4-420b-8427-72c23cfe6ed2 None None] MFA Rules not processed for user `c37a1ed5575d4f4ab7615ea38d6c47b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:27.624455 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6f5dd732-6ad4-420b-8427-72c23cfe6ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:27.684603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1425/2860] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:27.694723 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:27.696578 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:27.719763 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:27.760925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde4e7-f45e-4ddc-842e-9897ac8c6edb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:27.764206 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:27.764861 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:27.765154 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:27.765435 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:27.775373 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:27.776834 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:27.786631 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1436/2861] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:31.956078 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:31.956702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:31.956702 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:31.992046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] MFA Rules not processed for user `84f06063489847e68831b2c94fac2a26`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:31.993377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:32.013406 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:32.076182 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1426/2862] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:32.080700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:32.092720 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231119-6e84-4d35-bf13-8ae64f586b2e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:32.098519 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:32.099743 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:32.100385 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:32.100672 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:32.101393 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:32.103487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:32.138960 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1437/2863] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jul 21 22:43:32 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:43:32.142097 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:32.156527 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79bc6-5f11-497b-a9e2-3c6a9bce03bd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:32.224065 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:32.224840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:32.225128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:32.225413 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:32.226133 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:32.228106 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:32.253685 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1427/2864] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jul 21 22:43:32 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:43:32.258659 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:32.272642 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8118d10-b959-463d-b3e2-7c4a7c562095 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:32.277151 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:32.277816 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:32.278133 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:32.278416 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:32.279186 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:32.280817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:32.293376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1438/2865] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:43:32.297803 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:32.308408 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c0550-e479-4b2f-9b5a-b4eb0d6e92f9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:32.314058 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:32.314986 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:32.315321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:32.315651 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] PATH_INFO: `/v3/projects/fe9a471c5dbb4c0783c8174b6d9daf48/users/da4c2b5824ff48a08cdf61d6d3fa1800/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:32.337777 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9a471c5dbb4c0783c8174b6d9daf48, user_id=da4c2b5824ff48a08cdf61d6d3fa1800, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:32.341564 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:32.357187 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1428/2866] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:43:32 2022] PUT /identity/v3/projects/fe9a471c5dbb4c0783c8174b6d9daf48/users/da4c2b5824ff48a08cdf61d6d3fa1800/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:32.361020 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:32.372246 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6360e2a-3d0f-4a78-a255-ed47dbd75477 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:32.439143 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f0474-8c47-477f-a085-1ef2873784fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:32.439827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f0474-8c47-477f-a085-1ef2873784fd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:32.440119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f0474-8c47-477f-a085-1ef2873784fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:32.440398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f0474-8c47-477f-a085-1ef2873784fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:32.441139 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f0474-8c47-477f-a085-1ef2873784fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:32.443366 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f0474-8c47-477f-a085-1ef2873784fd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:32.453737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1439/2867] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jul 21 22:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:43:32.457945 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:32.469163 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781eefc3-99ad-4135-8962-460c0cc01e16 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:32.473886 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:32.474889 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:32.475200 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:32.475473 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] PATH_INFO: `/v3/projects/fe9a471c5dbb4c0783c8174b6d9daf48/users/da4c2b5824ff48a08cdf61d6d3fa1800/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:32.477448 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9a471c5dbb4c0783c8174b6d9daf48, user_id=da4c2b5824ff48a08cdf61d6d3fa1800, role_id=dbbfc4adb5f64ee3a84e8bb4eaa97f67)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:32.480103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:32.491068 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1429/2868] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jul 21 22:43:32 2022] PUT /identity/v3/projects/fe9a471c5dbb4c0783c8174b6d9daf48/users/da4c2b5824ff48a08cdf61d6d3fa1800/roles/dbbfc4adb5f64ee3a84e8bb4eaa97f67 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:43:36.930100 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa289-5dbf-413b-967a-246661ac188c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:36.931036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa289-5dbf-413b-967a-246661ac188c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:36.931036 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa289-5dbf-413b-967a-246661ac188c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:36.981192 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ef6aa289-5dbf-413b-967a-246661ac188c None None] MFA Rules not processed for user `da4c2b5824ff48a08cdf61d6d3fa1800`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:36.982736 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ef6aa289-5dbf-413b-967a-246661ac188c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:37.102398 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1440/2869] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:37.122648 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:37.126070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:37.232655 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c799e-0375-44e1-a6a8-64c611b02be4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:37.238911 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:37.239888 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:37.240277 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:37.240634 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:37.246925 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:37.316840 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:37.319602 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:37.330269 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1430/2870] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:38.672135 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:38.672594 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:38.672686 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:38.691706 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] MFA Rules not processed for user `84f06063489847e68831b2c94fac2a26`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:43:38.693251 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:38.700386 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1441/2871] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:43:47.326072 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=24 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:48.686376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:52.854491 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:52.855974 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:52.915112 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656445e-2bd4-4f9b-9b18-25f81bc8445c service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:52.918782 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd529f6af558a44ef909937d5a960cb88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:52.919468 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:52.919748 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:52.920123 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:52.922370 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:52.937595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:52.939043 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:52.951639 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1431/2872] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:53.698140 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:53.699719 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:53.721804 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:53.723562 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:53.774091 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389ff7-3036-4cf9-bf51-8f918109cf80 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:53.779219 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:53.779875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:53.780161 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:53.780434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:53.782305 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:53.782862 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f2e7-d893-4d7a-a9a5-fac2978c7473 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:53.786443 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:53.787710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:53.787998 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:53.788272 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:53.790007 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:53.795096 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:53.796716 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:53.804407 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:53.805849 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:53.807388 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1442/2873] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:43:53.820080 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1432/2874] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:43:55.779463 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:55.790675 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c3492-6ba7-4d2d-804d-40b947bef94e service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:55.799932 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:55.800761 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:55.801119 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:55.801470 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:55.804709 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:55.816110 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:55.817673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:55.827467 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1443/2875] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:56.469217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:56.475944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:56.511716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48571ffa-7c35-4039-8388-54d4a02f1211 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:56.518109 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:56.518854 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:56.519245 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:56.519603 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:56.521787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:43:56.537250 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:56.539149 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:56.551745 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1433/2876] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:05.823560 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=38 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:06.545882 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=58 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:06.991999 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:07.005946 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa2138-f07c-4283-a786-d0cf38b96896 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:07.011901 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:07.012558 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:07.012844 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:07.013378 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:07.015238 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:07.028424 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:07.030045 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f5f05-6716-4956-96f6-71f27a3e8a75 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:07.041665 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1444/2877] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:09.215329 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:09.224949 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817082e-4265-4357-a5a8-44549706fd43 service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:09.229171 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:09.230091 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:09.230377 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:09.230656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:09.239591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:09.241017 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:09.251452 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1434/2878] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:17.037144 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:19.247128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:19.792985 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:19.793537 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:19.793643 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:19.815907 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] MFA Rules not processed for user `a367906e74654507aaed8b885d525a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:19.817199 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:19.873405 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1445/2879] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:44:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:44:20.191116 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:20.199946 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b09d-53c5-4346-a44a-a59134f9200c service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:20.203710 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:20.204364 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:20.204673 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:20.204953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:20.206761 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:20.217122 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:20.218556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb89d61-e316-4c29-8f68-04e4a707ec34 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:20.226679 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1435/2880] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:29.865957 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=25 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:30.222899 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:02.366062 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:02.367505 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:02.382822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eba019-6378-496c-80ed-23a270c2b95b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:02.388810 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:02.389503 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:02.389791 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:02.390066 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:02.392285 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:02.404689 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:02.406245 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:02.416274 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1446/2881] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:12.412008 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:15.095497 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:15.096024 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:15.096135 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:15.114062 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] MFA Rules not processed for user `a367906e74654507aaed8b885d525a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:15.115656 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:15.122633 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1436/2882] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jul 21 22:46:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:16.238461 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:16.248231 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926249c1-366d-4f70-a9f3-e697e78a28d6 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:16.251935 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:16.252596 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:16.252886 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:16.253167 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:16.255001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:16.265673 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:16.267137 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:16.276164 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1447/2883] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:17.384287 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:17.396363 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3756eb7-e656-4a7d-bff2-d9a54f449d85 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:17.401037 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:17.401680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:17.401960 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:17.402234 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:17.404046 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:17.414253 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:17.415988 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:17.424381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1437/2884] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:17.528369 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:17.538279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761100c-0933-4ca6-91bc-14ba09b72038 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:17.542128 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:17.542979 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:17.543586 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:17.543875 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:17.545888 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:17.559833 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:17.561282 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:17.572698 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1448/2885] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:27.421715 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=9 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:27.567324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=12 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:42.140158 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:42.155321 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30bf2-08f8-45ef-a235-461265882b4b service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:42.161235 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '054c95ae7aa645b7ade6390e9bf0059f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:42.161857 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:42.162150 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:42.162435 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:42.165124 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:42.178001 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:42.180503 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:42.191977 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1438/2886] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:52.187266 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:52.484306 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:52.550474 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc97d7-82e9-42eb-a549-9880882a1540 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:52.555756 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:52.556639 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:52.556925 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:52.557201 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] PATH_INFO: `/v3/users/84f06063489847e68831b2c94fac2a26` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:52.558430 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f06063489847e68831b2c94fac2a26)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:52.560420 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:52.607962 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f06063489847e68831b2c94fac2a26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:52.615884 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f06063489847e68831b2c94fac2a26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:52.627973 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f06063489847e68831b2c94fac2a26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:52.628559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f06063489847e68831b2c94fac2a26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:52.635813 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f06063489847e68831b2c94fac2a26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:52.642501 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1449/2887] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:47:52 2022] DELETE /identity/v3/users/84f06063489847e68831b2c94fac2a26 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:47:52.817285 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:52.828781 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26902b9-7879-49de-9a51-0ae83d058be2 service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:52.889022 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:52.889880 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:52.890103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:52.890381 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] PATH_INFO: `/v3/projects/8323020521c94131b7d9bef5aa456a70` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:52.891595 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8323020521c94131b7d9bef5aa456a70)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:52.893525 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:53.064121 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8323020521c94131b7d9bef5aa456a70', 'request_id': 'req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.072600 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8323020521c94131b7d9bef5aa456a70', 'request_id': 'req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.079241 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8323020521c94131b7d9bef5aa456a70', 'request_id': 'req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.089967 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] The token cache is being invalidate because project 8323020521c94131b7d9bef5aa456a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:47:53.090263 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.090912 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.092490 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1439/2888] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:47:52 2022] DELETE /identity/v3/projects/8323020521c94131b7d9bef5aa456a70 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:47:53.096589 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:53.098061 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:53.111571 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76163a52-8891-4454-a421-08256b22e071 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:53.168101 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:53.168893 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:53.169177 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:53.169456 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] PATH_INFO: `/v3/users/da4c2b5824ff48a08cdf61d6d3fa1800` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:53.170656 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4c2b5824ff48a08cdf61d6d3fa1800)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:53.172658 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:53.195376 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:53.220086 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4c2b5824ff48a08cdf61d6d3fa1800'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.232597 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4c2b5824ff48a08cdf61d6d3fa1800'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.245740 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4c2b5824ff48a08cdf61d6d3fa1800'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.246427 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4c2b5824ff48a08cdf61d6d3fa1800'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.253466 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4c2b5824ff48a08cdf61d6d3fa1800'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.260320 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1450/2889] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:47:53 2022] DELETE /identity/v3/users/da4c2b5824ff48a08cdf61d6d3fa1800 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:47:53.442400 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:53.443997 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:53.458613 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bc0f6-bb7d-437e-8844-2f6c1060cb8c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:53.520714 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:53.522215 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:53.522556 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:53.522837 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] PATH_INFO: `/v3/projects/fe9a471c5dbb4c0783c8174b6d9daf48` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:53.524103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9a471c5dbb4c0783c8174b6d9daf48)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:53.526018 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:53.565231 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:53.580471 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9a471c5dbb4c0783c8174b6d9daf48', 'request_id': 'req-37d6f121-b3e2-436e-ac39-520986e08042'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.591476 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9a471c5dbb4c0783c8174b6d9daf48', 'request_id': 'req-37d6f121-b3e2-436e-ac39-520986e08042'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.604209 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9a471c5dbb4c0783c8174b6d9daf48', 'request_id': 'req-37d6f121-b3e2-436e-ac39-520986e08042'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.616810 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] The token cache is being invalidate because project fe9a471c5dbb4c0783c8174b6d9daf48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:47:53.617103 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.617787 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:47:53.619376 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1440/2890] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:47:53 2022] DELETE /identity/v3/projects/fe9a471c5dbb4c0783c8174b6d9daf48 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:48:03.255921 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:03.612217 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:16.477883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:16.479351 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:16.541279 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613e81d-9840-4ef1-a118-63d72268ba79 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:16.588324 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:16.588883 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:16.589162 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:16.589434 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:16.591255 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:16.649117 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:16.650559 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:16.659533 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1451/2891] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:48:26.655381 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=68 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:30.005666 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:30.007128 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:30.020000 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6f121-b3e2-436e-ac39-520986e08042 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:30.024169 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5dceea71d1434796a105ea67773cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:30.024811 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:30.025087 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:30.025357 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:30.027152 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:30.091865 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:30.093299 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:30.102894 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1441/2892] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:48:33.215895 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:33.217822 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:33.306617 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c0b41-6779-4e1a-a401-6e59232ae177 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:33.311867 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:33.312527 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:33.312817 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:33.313108 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:33.314914 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:33.329001 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:33.330967 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:33.343367 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1452/2893] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:48:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:48:36.228084 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:36.229540 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:36.243551 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a46f-c0ae-4b73-8291-b2ed9e5b9134 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:36.248178 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:36.249040 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:36.249319 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:36.249591 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] PATH_INFO: `/v3/users/a367906e74654507aaed8b885d525a0f` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:36.250762 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a367906e74654507aaed8b885d525a0f)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:36.252716 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:36.308823 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a367906e74654507aaed8b885d525a0f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.316114 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a367906e74654507aaed8b885d525a0f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.316953 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a367906e74654507aaed8b885d525a0f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.324680 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a367906e74654507aaed8b885d525a0f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.337353 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a367906e74654507aaed8b885d525a0f'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.343419 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1442/2894] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:48:36 2022] DELETE /identity/v3/users/a367906e74654507aaed8b885d525a0f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:48:36.518487 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:36.531126 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404e710-9a37-47ea-b617-e0e0ef0f3e78 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:36.591229 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:36.592168 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:36.592453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:36.592730 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] PATH_INFO: `/v3/projects/e61e273f945e45f0a6e1f3b45e4bc9d2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:36.593947 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e61e273f945e45f0a6e1f3b45e4bc9d2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:36.595965 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:36.644402 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e61e273f945e45f0a6e1f3b45e4bc9d2', 'request_id': 'req-3c13f00a-9070-494d-8b38-0f9ddc8e823a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.650785 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e61e273f945e45f0a6e1f3b45e4bc9d2', 'request_id': 'req-3c13f00a-9070-494d-8b38-0f9ddc8e823a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.656303 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e61e273f945e45f0a6e1f3b45e4bc9d2', 'request_id': 'req-3c13f00a-9070-494d-8b38-0f9ddc8e823a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.671004 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] The token cache is being invalidate because project e61e273f945e45f0a6e1f3b45e4bc9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:48:36.671317 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.671928 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.673453 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1453/2895] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:48:36 2022] DELETE /identity/v3/projects/e61e273f945e45f0a6e1f3b45e4bc9d2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:48:36.677531 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:36.679035 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:36.693333 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd6a8b-84ca-4ec8-9b9d-92a4dc9372cd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:36.750999 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:36.751902 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:36.752184 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:36.752458 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] PATH_INFO: `/v3/users/46bce9df7ec449389efe4aa68ecd9b03` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:36.753653 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46bce9df7ec449389efe4aa68ecd9b03)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:36.755583 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:36.761900 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:36.809306 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46bce9df7ec449389efe4aa68ecd9b03'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.816944 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46bce9df7ec449389efe4aa68ecd9b03'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.817686 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46bce9df7ec449389efe4aa68ecd9b03'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.825134 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46bce9df7ec449389efe4aa68ecd9b03'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.837980 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46bce9df7ec449389efe4aa68ecd9b03'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:36.844347 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1443/2896] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jul 21 22:48:36 2022] DELETE /identity/v3/users/46bce9df7ec449389efe4aa68ecd9b03 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:48:37.020702 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:48:37.023649 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:48:37.045492 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f00a-9070-494d-8b38-0f9ddc8e823a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:48:37.062520 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:37.106014 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c37a1ed5575d4f4ab7615ea38d6c47b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a2e5fa7e325464b9d8cfafd39af8ec6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:48:37.106882 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:48:37.107215 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:48:37.107497 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] PATH_INFO: `/v3/projects/1fe0b11f114e440582f0edbe0614caa9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:48:37.108814 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe0b11f114e440582f0edbe0614caa9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:48:37.110815 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:48:37.157827 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe0b11f114e440582f0edbe0614caa9', 'request_id': 'req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:37.163939 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe0b11f114e440582f0edbe0614caa9', 'request_id': 'req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:37.169566 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe0b11f114e440582f0edbe0614caa9', 'request_id': 'req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:37.184235 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] The token cache is being invalidate because project 1fe0b11f114e440582f0edbe0614caa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:48:37.184530 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:37.185169 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa48fc7-8f2a-4be1-a334-c21ecf4719f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:48:37.186700 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1454/2897] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jul 21 22:48:37 2022] DELETE /identity/v3/projects/1fe0b11f114e440582f0edbe0614caa9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:48:46.840405 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:47.179737 dv-cinder-850698-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:07.358582 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:07.360070 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:07.506747 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c850f-8c72-49b6-9b00-dc34008e4b61 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:07.565324 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c423e880647139535f2d0dfc5a9c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9e65f853d5348279e8ec6b19ee9773a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:07.565970 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:07.566248 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:07.566522 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:07.568423 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:07.580228 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:07.581779 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa2d02-44cd-4a57-9906-a89553995d56 service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:07.592185 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1444/2898] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:17.587293 dv-cinder-850698-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}